[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:17:43 dist-test-slave-5k54 krb5kdc[8539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:17:43 dist-test-slave-5k54 krb5kdc[8539](info): set up 2 sockets
Aug 12 14:17:43 dist-test-slave-5k54 krb5kdc[8539](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 14:17:44.659188 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:17:44 dist-test-slave-5k54 krb5kdc[8539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:17:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:17:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:17:44.787029 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:33697
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:34945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:33697
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:44.881841 8556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:44.882073 8556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:44.882112 8556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:44.883272 8556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:17:44.883334 8556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:44.883361 8556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:44.883397 8556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:17:44.883458 8556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:17:44.885095 8556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:34945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:33697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:33697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:44.885545 8556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:44.885838 8556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:44.893138 8565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:44.893754 8562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:44.894232 8563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:44.894466 8556 server_base.cc:1034] running on GCE node
I20260812 14:17:44.894778 8556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:44.895242 8556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:44.896405 8556 hybrid_clock.cc:648] HybridClock initialized: now 1786544264896367 us; error 67 us; skew 500 ppm
Aug 12 14:17:44 dist-test-slave-5k54 krb5kdc[8539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544264, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:44.899062 8556 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:17:44.899722 8556 webserver.cc:460] Webserver started at http://127.8.85.62:36171/ using document root <none> and password file <none>
I20260812 14:17:44.900025 8556 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:44.900072 8556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:44.900202 8556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:44.901487 8556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "5f62be614c494b9893fd6fdca4d8f5c0"
format_stamp: "Formatted at 2026-08-12 14:17:44 on dist-test-slave-5k54"
server_key: "e716c31f71e440f4588e99a986581a2e"
server_key_iv: "b8d6ae39003ccf24e55f9e27fae8c6be"
server_key_version: "encryptionkey@0"
I20260812 14:17:44.902140 8556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "5f62be614c494b9893fd6fdca4d8f5c0"
format_stamp: "Formatted at 2026-08-12 14:17:44 on dist-test-slave-5k54"
server_key: "e716c31f71e440f4588e99a986581a2e"
server_key_iv: "b8d6ae39003ccf24e55f9e27fae8c6be"
server_key_version: "encryptionkey@0"
I20260812 14:17:44.904148 8556 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 14:17:44.905148 8573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:44.905398 8556 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:17:44.905496 8556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "5f62be614c494b9893fd6fdca4d8f5c0"
format_stamp: "Formatted at 2026-08-12 14:17:44 on dist-test-slave-5k54"
server_key: "e716c31f71e440f4588e99a986581a2e"
server_key_iv: "b8d6ae39003ccf24e55f9e27fae8c6be"
server_key_version: "encryptionkey@0"
I20260812 14:17:44.905576 8556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:44.930966 8556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:44.933120 8556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:44.933321 8556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:44.939056 8556 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:33697
I20260812 14:17:44.939164 8634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:33697 every 8 connection(s)
I20260812 14:17:44.939787 8556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:17:44.941440 8636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:44.944075 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 8556
I20260812 14:17:44.944172 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:17:44.944903 8532 external_mini_cluster.cc:1428] Setting key cd3ce9355bce6ade72a4b383ac723004
I20260812 14:17:44.945098 8636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0: Bootstrap starting.
I20260812 14:17:44.946386 8636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:44.946924 8636 log.cc:824] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0: Log is configured to *not* fsync() on all Append() calls
I20260812 14:17:44.947808 8636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0: No bootstrap required, opened a new log
I20260812 14:17:44.949891 8636 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62be614c494b9893fd6fdca4d8f5c0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33697 } }
I20260812 14:17:44.950146 8636 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:44.950259 8636 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f62be614c494b9893fd6fdca4d8f5c0, State: Initialized, Role: FOLLOWER
I20260812 14:17:44.950466 8636 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62be614c494b9893fd6fdca4d8f5c0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33697 } }
I20260812 14:17:44.950611 8636 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:17:44.950660 8636 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:17:44.950728 8636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:44.951795 8636 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62be614c494b9893fd6fdca4d8f5c0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33697 } }
I20260812 14:17:44.951985 8636 leader_election.cc:302] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f62be614c494b9893fd6fdca4d8f5c0; no voters:
I20260812 14:17:44.952236 8636 leader_election.cc:288] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:17:44.952260 8641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:17:44.952452 8641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [term 1 LEADER]: Becoming Leader. State: Replica: 5f62be614c494b9893fd6fdca4d8f5c0, State: Running, Role: LEADER
I20260812 14:17:44.952677 8641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62be614c494b9893fd6fdca4d8f5c0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33697 } }
I20260812 14:17:44.952813 8636 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:17:44.953377 8641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f62be614c494b9893fd6fdca4d8f5c0. Latest consensus state: current_term: 1 leader_uuid: "5f62be614c494b9893fd6fdca4d8f5c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62be614c494b9893fd6fdca4d8f5c0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33697 } } }
I20260812 14:17:44.953862 8641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [sys.catalog]: This master's current role is: LEADER
I20260812 14:17:44.953768 8642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f62be614c494b9893fd6fdca4d8f5c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f62be614c494b9893fd6fdca4d8f5c0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33697 } } }
I20260812 14:17:44.954368 8642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0 [sys.catalog]: This master's current role is: LEADER
W20260812 14:17:44.955803 8660 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:17:44.955868 8660 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:17:44.955988 8651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:17:44.956317 8651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:17:44.958993 8651 catalog_manager.cc:1347] Generated new cluster ID: 2098d1faa0c74c6089234e97c8c37e91
I20260812 14:17:44.959044 8651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:17:44 dist-test-slave-5k54 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:44.967208 8638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:44.946436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49646 (local address 127.8.85.62:33697)
0812 14:17:44.946941 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:44.946954 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:44.947153 (+ 199us) server_negotiation.cc:407] Connection header received
0812 14:17:44.949848 (+ 2695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:44.949855 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:44.949962 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:44.950138 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:44.952266 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:44.954311 (+ 2045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:44.956524 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:44.957329 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:44.957494 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:44.962127 (+ 4633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:44.962131 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:44.962131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:44.962155 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:44.964301 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:44.964860 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:44.964863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:44.964863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:44.965008 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:44.965405 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:44.965407 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:44.965407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:44.965788 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:44.966647 (+ 859us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:44.966744 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 14:17:44.966950 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":101,"threads_started":1}
I20260812 14:17:44.981400 8651 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:17:44.982204 8651 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:17:44.991920 8651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f62be614c494b9893fd6fdca4d8f5c0: Generated new TSK 0
I20260812 14:17:44.992136 8651 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:17:45.025935 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33697
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:34945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:45.124081 8667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:45.124344 8667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:45.124387 8667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:45.125664 8667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:45.125733 8667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:45.125788 8667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:17:45.127321 8667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:34945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33697
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:45.127665 8667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:45.127908 8667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:45.133515 8672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:45.133795 8673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:45.133816 8667 server_base.cc:1034] running on GCE node
W20260812 14:17:45.133806 8676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:45.134192 8667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:45.134434 8667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:45.135592 8667 hybrid_clock.cc:648] HybridClock initialized: now 1786544265135571 us; error 37 us; skew 500 ppm
Aug 12 14:17:45 dist-test-slave-5k54 krb5kdc[8539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544265, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:45.138136 8667 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:17:45.138579 8667 webserver.cc:460] Webserver started at http://127.8.85.1:34935/ using document root <none> and password file <none>
I20260812 14:17:45.138767 8667 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:45.138810 8667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:45.138900 8667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:45.140089 8667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "90ba1aee40a245f1b261e6ad82b07bea"
format_stamp: "Formatted at 2026-08-12 14:17:45 on dist-test-slave-5k54"
server_key: "cbf6f29279027dc11fbf1a3e3e4096b7"
server_key_iv: "f1761104ed7d06708e3dc255ba9ce124"
server_key_version: "encryptionkey@0"
I20260812 14:17:45.140455 8667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "90ba1aee40a245f1b261e6ad82b07bea"
format_stamp: "Formatted at 2026-08-12 14:17:45 on dist-test-slave-5k54"
server_key: "cbf6f29279027dc11fbf1a3e3e4096b7"
server_key_iv: "f1761104ed7d06708e3dc255ba9ce124"
server_key_version: "encryptionkey@0"
I20260812 14:17:45.142020 8667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:17:45.142750 8684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:45.142940 8667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:17:45.143031 8667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "90ba1aee40a245f1b261e6ad82b07bea"
format_stamp: "Formatted at 2026-08-12 14:17:45 on dist-test-slave-5k54"
server_key: "cbf6f29279027dc11fbf1a3e3e4096b7"
server_key_iv: "f1761104ed7d06708e3dc255ba9ce124"
server_key_version: "encryptionkey@0"
I20260812 14:17:45.143105 8667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:45.166906 8667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:45.169345 8667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:45.169559 8667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:45.169962 8667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:17:45.170364 8667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:17:45.170457 8667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:45.170482 8667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:17:45.170526 8667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:45.178174 8667 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:46065
I20260812 14:17:45.178258 8815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:46065 every 8 connection(s)
I20260812 14:17:45.178594 8667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:17:45.182447 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 8667
I20260812 14:17:45.182554 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:17:45.182708 8532 external_mini_cluster.cc:1428] Setting key e1dcd8b8532857eb35953014146abc9d
Aug 12 14:17:45 dist-test-slave-5k54 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544265, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:45.189317 8638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:45.179680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:45387 (local address 127.8.85.62:33697)
0812 14:17:45.179757 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:45.179759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:45.179921 (+ 162us) server_negotiation.cc:407] Connection header received
0812 14:17:45.180458 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:45.180459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:45.180475 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:45.180524 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:45.181706 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.182323 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.183824 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.184141 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.184233 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:45.186456 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:45.186458 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:45.186458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:45.186466 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:45.187915 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:45.188334 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:45.188335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:45.188336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:45.188399 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:45.188648 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:45.188649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:45.188650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:45.188766 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:45.188839 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:45.189119 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 14:17:45.189223 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 14:17:45.189491 8818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:45.179604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33697 (local address 127.8.85.1:45387)
0812 14:17:45.179787 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:45.179831 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:45.180367 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:45.180600 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:45.180604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:45.181398 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:45.181612 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:45.181618 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.182430 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.182431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:45.183731 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:45.183734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.184246 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.184247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:45.184390 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:45.184942 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:45.184951 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:45.186348 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:45.188016 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:45.188024 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:45.188026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:45.188257 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:45.188471 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:45.188471 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:45.188472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:45.188586 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:45.188830 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:45.188838 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:45.188981 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 14:17:45.189082 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 14:17:45.189246 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260812 14:17:45.190147 8816 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33697
I20260812 14:17:45.190265 8816 heartbeater.cc:464] Registering TS with master...
I20260812 14:17:45.190478 8816 heartbeater.cc:511] Master 127.8.85.62:33697 requested a full tablet report, sending...
I20260812 14:17:45.191515 8593 ts_manager.cc:194] Registered new tserver with Master: 90ba1aee40a245f1b261e6ad82b07bea (127.8.85.1:46065)
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
I20260812 14:17:45.192314 8593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:45387
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:17:45.211063 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33697
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:34945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:45.294521 8822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:45.294716 8822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:45.294741 8822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:45.295997 8822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:45.296042 8822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:45.296070 8822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:17:45.297456 8822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:34945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33697
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:45.297643 8822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:45.297819 8822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:45.302912 8834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:45.302973 8822 server_base.cc:1034] running on GCE node
W20260812 14:17:45.302994 8830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:45.303123 8831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:45.303323 8822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:45.303644 8822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:45.304807 8822 hybrid_clock.cc:648] HybridClock initialized: now 1786544265304798 us; error 34 us; skew 500 ppm
Aug 12 14:17:45 dist-test-slave-5k54 krb5kdc[8539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544265, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:45.307358 8822 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:17:45.307816 8822 webserver.cc:460] Webserver started at http://127.8.85.2:39719/ using document root <none> and password file <none>
I20260812 14:17:45.308079 8822 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:45.308125 8822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:45.308291 8822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:45.309571 8822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "71a88b752c824461bfd4c9d4f0016ebf"
format_stamp: "Formatted at 2026-08-12 14:17:45 on dist-test-slave-5k54"
server_key: "6a54d571ca6a18ae80f73de2f2262691"
server_key_iv: "3fe9f3e941c471418b1908ce93065d5b"
server_key_version: "encryptionkey@0"
I20260812 14:17:45.309932 8822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "71a88b752c824461bfd4c9d4f0016ebf"
format_stamp: "Formatted at 2026-08-12 14:17:45 on dist-test-slave-5k54"
server_key: "6a54d571ca6a18ae80f73de2f2262691"
server_key_iv: "3fe9f3e941c471418b1908ce93065d5b"
server_key_version: "encryptionkey@0"
I20260812 14:17:45.311401 8822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:17:45.312162 8844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:45.312330 8822 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:17:45.312429 8822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "71a88b752c824461bfd4c9d4f0016ebf"
format_stamp: "Formatted at 2026-08-12 14:17:45 on dist-test-slave-5k54"
server_key: "6a54d571ca6a18ae80f73de2f2262691"
server_key_iv: "3fe9f3e941c471418b1908ce93065d5b"
server_key_version: "encryptionkey@0"
I20260812 14:17:45.312497 8822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:45.328106 8822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:45.330432 8822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:45.330576 8822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:45.330807 8822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:17:45.331116 8822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:17:45.331151 8822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:45.331189 8822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:17:45.331216 8822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:45.338009 8822 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:44413
I20260812 14:17:45.338054 8975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:44413 every 8 connection(s)
I20260812 14:17:45.338364 8822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:17:45 dist-test-slave-5k54 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544265, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:45.346431 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 8822
I20260812 14:17:45.346532 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:17:45.346732 8532 external_mini_cluster.cc:1428] Setting key 407eff5be0403284aadd17c8d80c0cbb
I20260812 14:17:45.347714 8638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:45.339005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:50003 (local address 127.8.85.62:33697)
0812 14:17:45.339072 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:45.339073 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:45.339307 (+ 234us) server_negotiation.cc:407] Connection header received
0812 14:17:45.339865 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:45.339867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:45.339880 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:45.339929 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:45.340858 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.341379 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.342387 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.342597 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.342671 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:45.344647 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:45.344648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:45.344648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:45.344655 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:45.346149 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:45.346679 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:45.346680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:45.346680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:45.346733 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:45.347026 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:45.347027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:45.347028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:45.347170 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:45.347237 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:45.347560 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 14:17:45.347648 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:17:45.347971 8978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:45.338971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33697 (local address 127.8.85.2:50003)
0812 14:17:45.339154 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:45.339212 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:45.339737 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:45.340016 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:45.340019 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:45.340580 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:45.340762 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:45.340766 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.341473 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.341474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:45.342274 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:45.342276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.342676 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.342677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:45.342807 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:45.343222 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:45.343230 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:45.344562 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:45.346250 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:45.346256 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:45.346258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:45.346601 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:45.346803 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:45.346803 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:45.346804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:45.346946 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:45.347234 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:45.347239 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:45.347394 (+ 155us) client_negotiation.cc:769] Sending connection context
0812 14:17:45.347522 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 14:17:45.347711 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
I20260812 14:17:45.348491 8976 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33697
I20260812 14:17:45.348596 8976 heartbeater.cc:464] Registering TS with master...
I20260812 14:17:45.348767 8976 heartbeater.cc:511] Master 127.8.85.62:33697 requested a full tablet report, sending...
I20260812 14:17:45.349123 8593 ts_manager.cc:194] Registered new tserver with Master: 71a88b752c824461bfd4c9d4f0016ebf (127.8.85.2:44413)
I20260812 14:17:45.349504 8593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:50003
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:17:45.374269 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33697
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:34945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:45.460932 8982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:45.461134 8982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:45.461158 8982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:45.462239 8982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:45.462276 8982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:45.462302 8982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:17:45.463745 8982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:34945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33697
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:45.463929 8982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:45.464131 8982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:45.469416 8993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:45.469352 8989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:45.469389 8988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:45.469377 8982 server_base.cc:1034] running on GCE node
I20260812 14:17:45.469892 8982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:45.470161 8982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:45.471329 8982 hybrid_clock.cc:648] HybridClock initialized: now 1786544265471319 us; error 36 us; skew 500 ppm
Aug 12 14:17:45 dist-test-slave-5k54 krb5kdc[8539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544265, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:45.473892 8982 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:17:45.474370 8982 webserver.cc:460] Webserver started at http://127.8.85.3:35421/ using document root <none> and password file <none>
I20260812 14:17:45.474592 8982 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:45.474656 8982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:45.474802 8982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:45.476012 8982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "62971acd37a34873b04697d0eddb3e3e"
format_stamp: "Formatted at 2026-08-12 14:17:45 on dist-test-slave-5k54"
server_key: "8c4d84bcad2380ea8eae71e43b76997b"
server_key_iv: "196d0cf22ce336f91e04e5fc84cbe808"
server_key_version: "encryptionkey@0"
I20260812 14:17:45.476450 8982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "62971acd37a34873b04697d0eddb3e3e"
format_stamp: "Formatted at 2026-08-12 14:17:45 on dist-test-slave-5k54"
server_key: "8c4d84bcad2380ea8eae71e43b76997b"
server_key_iv: "196d0cf22ce336f91e04e5fc84cbe808"
server_key_version: "encryptionkey@0"
I20260812 14:17:45.477782 8982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:17:45.478576 9002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:45.478744 8982 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:17:45.478879 8982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "62971acd37a34873b04697d0eddb3e3e"
format_stamp: "Formatted at 2026-08-12 14:17:45 on dist-test-slave-5k54"
server_key: "8c4d84bcad2380ea8eae71e43b76997b"
server_key_iv: "196d0cf22ce336f91e04e5fc84cbe808"
server_key_version: "encryptionkey@0"
I20260812 14:17:45.478964 8982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:45.490592 8982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:45.492977 8982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:45.493139 8982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:45.493353 8982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:17:45.493662 8982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:17:45.493696 8982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:45.493734 8982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:17:45.493760 8982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:45.500181 8982 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:35317
I20260812 14:17:45.500255 9133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:35317 every 8 connection(s)
I20260812 14:17:45.500650 8982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:17:45 dist-test-slave-5k54 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544265, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:45.509774 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 8982
I20260812 14:17:45.509863 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:17:45.510021 8532 external_mini_cluster.cc:1428] Setting key a667ae968709aac0a4845bce115cb351
I20260812 14:17:45.510382 8638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:45.501493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:50033 (local address 127.8.85.62:33697)
0812 14:17:45.501561 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:45.501563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:45.501785 (+ 222us) server_negotiation.cc:407] Connection header received
0812 14:17:45.502410 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:45.502411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:45.502422 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:45.502476 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:45.503536 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.504048 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.505152 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.505352 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.505411 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:45.507552 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:45.507553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:45.507554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:45.507561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:45.509026 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:45.509381 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:45.509382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:45.509382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:45.509431 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:45.509644 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:45.509645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:45.509646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:45.509787 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:45.509905 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:45.510206 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 14:17:45.510293 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:17:45.510584 9137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:45.501470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33697 (local address 127.8.85.3:50033)
0812 14:17:45.501661 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:45.501700 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:45.502313 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:45.502558 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:45.502562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:45.503177 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:45.503400 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:45.503405 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.504141 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.504142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:45.505062 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:45.505065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.505432 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.505433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:45.505542 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:45.506006 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:45.506015 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:45.507400 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:45.509102 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:45.509108 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:45.509110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:45.509318 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:45.509485 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:45.509486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:45.509486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:45.509582 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:45.509871 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:45.509878 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:45.510030 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 14:17:45.510164 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 14:17:45.510328 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":104,"threads_started":1}
I20260812 14:17:45.511103 9134 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33697
I20260812 14:17:45.511225 9134 heartbeater.cc:464] Registering TS with master...
I20260812 14:17:45.511413 9134 heartbeater.cc:511] Master 127.8.85.62:33697 requested a full tablet report, sending...
I20260812 14:17:45.511819 8593 ts_manager.cc:194] Registered new tserver with Master: 62971acd37a34873b04697d0eddb3e3e (127.8.85.3:35317)
I20260812 14:17:45.512188 8593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:50033
I20260812 14:17:45.522377 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 14:17:45 dist-test-slave-5k54 krb5kdc[8539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544265, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 14:17:45 dist-test-slave-5k54 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544265, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.85.1@KRBTEST.COM
I20260812 14:17:45.547549 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:45.539695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40092 (local address 127.8.85.1:46065)
0812 14:17:45.539871 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:45.539878 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:45.539895 (+ 17us) server_negotiation.cc:407] Connection header received
0812 14:17:45.539922 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:45.539924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:45.539939 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:45.540006 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:45.540547 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.541272 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.542014 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.542341 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.542407 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:45.544688 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:45.544696 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:45.544696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:45.544703 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:45.546352 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:45.546720 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:45.546723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:45.546723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:45.546783 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:45.547019 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:45.547020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:45.547020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:45.547165 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:45.547269 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:45.547339 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 14:17:45.547415 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":101,"threads_started":1}
W20260812 14:17:45.547992 8719 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40092
Aug 12 14:17:45 dist-test-slave-5k54 krb5kdc[8539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 14:17:45 dist-test-slave-5k54 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.1@KRBTEST.COM
I20260812 14:17:45.570815 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:45.563969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40100 (local address 127.8.85.1:46065)
0812 14:17:45.564034 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:45.564036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:45.564075 (+ 39us) server_negotiation.cc:407] Connection header received
0812 14:17:45.564128 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:45.564128 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:45.564139 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:45.564187 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:45.564714 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.565276 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.565959 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:45.566178 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:45.566234 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:45.568334 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:45.568336 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:45.568336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:45.568344 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:45.569716 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:45.570067 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:45.570068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:45.570069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:45.570125 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:45.570316 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:45.570317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:45.570317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:45.570414 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:45.570531 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:45.570631 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 14:17:45.570710 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 14:17:45.571391 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 8667
I20260812 14:17:45.580623 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 8822
I20260812 14:17:45.591050 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 8982
I20260812 14:17:45.599004 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 8556
2026-08-12T14:17:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2088 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:17:45 dist-test-slave-5k54 krb5kdc[9154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:17:45 dist-test-slave-5k54 krb5kdc[9154](info): set up 2 sockets
Aug 12 14:17:45 dist-test-slave-5k54 krb5kdc[9154](info): commencing operation
krb5kdc: starting...
W20260812 14:17:46.708488 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:17:46 dist-test-slave-5k54 krb5kdc[9154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:17:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:17:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:17:46.830349 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:39555
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:39555
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:46.915280 9170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:46.915499 9170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:46.915522 9170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:46.916678 9170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:17:46.916719 9170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:46.916734 9170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:46.916746 9170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:17:46.916792 9170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:17:46.918246 9170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:39555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:39555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:46.918490 9170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:46.918727 9170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:46.923507 9176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:46.923525 9175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:46.923475 9178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:46.923734 9170 server_base.cc:1034] running on GCE node
I20260812 14:17:46.923985 9170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:46.924291 9170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:46.925444 9170 hybrid_clock.cc:648] HybridClock initialized: now 1786544266925425 us; error 30 us; skew 500 ppm
Aug 12 14:17:46 dist-test-slave-5k54 krb5kdc[9154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544266, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:46.927929 9170 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:17:46.928311 9170 webserver.cc:460] Webserver started at http://127.8.85.62:35333/ using document root <none> and password file <none>
I20260812 14:17:46.928519 9170 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:46.928558 9170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:46.928643 9170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:46.929632 9170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "cbaa7e5a5e284d609b7a55f453dc533c"
format_stamp: "Formatted at 2026-08-12 14:17:46 on dist-test-slave-5k54"
server_key: "220e819a70b9d2b6c205f2cecdcce45a"
server_key_iv: "49c0fb911524a05373810096d0283a34"
server_key_version: "encryptionkey@0"
I20260812 14:17:46.929927 9170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "cbaa7e5a5e284d609b7a55f453dc533c"
format_stamp: "Formatted at 2026-08-12 14:17:46 on dist-test-slave-5k54"
server_key: "220e819a70b9d2b6c205f2cecdcce45a"
server_key_iv: "49c0fb911524a05373810096d0283a34"
server_key_version: "encryptionkey@0"
I20260812 14:17:46.931210 9170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:17:46.931983 9186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:46.932137 9170 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:46.932227 9170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "cbaa7e5a5e284d609b7a55f453dc533c"
format_stamp: "Formatted at 2026-08-12 14:17:46 on dist-test-slave-5k54"
server_key: "220e819a70b9d2b6c205f2cecdcce45a"
server_key_iv: "49c0fb911524a05373810096d0283a34"
server_key_version: "encryptionkey@0"
I20260812 14:17:46.932284 9170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:46.949962 9170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:46.952205 9170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:46.952322 9170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:46.956272 9170 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:39555
I20260812 14:17:46.956343 9245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:39555 every 8 connection(s)
I20260812 14:17:46.956673 9170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:17:46.957664 9246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:46.960013 9246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c: Bootstrap starting.
I20260812 14:17:46.960646 9246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:46.960949 9246 log.cc:824] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:17:46.961616 9246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c: No bootstrap required, opened a new log
I20260812 14:17:46.963219 9246 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbaa7e5a5e284d609b7a55f453dc533c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 39555 } }
I20260812 14:17:46.963375 9246 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:46.963449 9246 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbaa7e5a5e284d609b7a55f453dc533c, State: Initialized, Role: FOLLOWER
I20260812 14:17:46.963609 9246 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbaa7e5a5e284d609b7a55f453dc533c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 39555 } }
I20260812 14:17:46.963690 9246 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:17:46.963729 9246 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:17:46.963780 9246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:46.964294 9246 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbaa7e5a5e284d609b7a55f453dc533c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 39555 } }
I20260812 14:17:46.964411 9246 leader_election.cc:302] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbaa7e5a5e284d609b7a55f453dc533c; no voters:
I20260812 14:17:46.964591 9246 leader_election.cc:288] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:17:46.964632 9249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:17:46.964869 9246 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:17:46.965306 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 9170
I20260812 14:17:46.965401 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:17:46.965569 8532 external_mini_cluster.cc:1428] Setting key 0824abb05a93f89ce82fd8e4e7e6ce70
I20260812 14:17:46.965754 9249 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [term 1 LEADER]: Becoming Leader. State: Replica: cbaa7e5a5e284d609b7a55f453dc533c, State: Running, Role: LEADER
I20260812 14:17:46.965865 9249 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbaa7e5a5e284d609b7a55f453dc533c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 39555 } }
I20260812 14:17:46.966367 9251 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbaa7e5a5e284d609b7a55f453dc533c. Latest consensus state: current_term: 1 leader_uuid: "cbaa7e5a5e284d609b7a55f453dc533c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbaa7e5a5e284d609b7a55f453dc533c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 39555 } } }
I20260812 14:17:46.966516 9250 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbaa7e5a5e284d609b7a55f453dc533c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbaa7e5a5e284d609b7a55f453dc533c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 39555 } } }
I20260812 14:17:46.966641 9251 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [sys.catalog]: This master's current role is: LEADER
I20260812 14:17:46.966645 9250 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c [sys.catalog]: This master's current role is: LEADER
W20260812 14:17:46.966868 9271 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:17:46.966925 9271 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:17:46.966995 9273 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:17:46.967157 9273 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:17:46.968978 9273 catalog_manager.cc:1347] Generated new cluster ID: 7a5eb3e27fd642e19bc1babd6fa53d53
I20260812 14:17:46.969036 9273 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:17:46 dist-test-slave-5k54 krb5kdc[9154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:46.978041 9269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:46.966130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54586 (local address 127.8.85.62:39555)
0812 14:17:46.966680 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:46.966755 (+ 75us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:46.966791 (+ 36us) server_negotiation.cc:407] Connection header received
0812 14:17:46.966929 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:46.966936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:46.966994 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:46.967194 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:46.968039 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:46.969217 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:46.969966 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:46.970393 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:46.970508 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:46.974562 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:46.974566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:46.974566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:46.974579 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:46.976486 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:46.976845 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:46.976848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:46.976849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:46.976951 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:46.977157 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:46.977158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:46.977158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:46.977415 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:46.977596 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:46.977667 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 14:17:46.977812 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":73,"threads_started":1}
I20260812 14:17:46.979967 9273 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:17:46.980650 9273 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:17:46.988958 9273 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbaa7e5a5e284d609b7a55f453dc533c: Generated new TSK 0
I20260812 14:17:46.989140 9273 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:17:47.026335 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:39555
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:47.123870 9278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:47.124073 9278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:47.124097 9278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:47.125278 9278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:47.125329 9278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:47.125355 9278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:17:47.126863 9278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:39555
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:47.127121 9278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:47.127354 9278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:47.132342 9284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:47.132350 9285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:47.132350 9287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:47.132786 9278 server_base.cc:1034] running on GCE node
I20260812 14:17:47.133107 9278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:47.133322 9278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:47.134466 9278 hybrid_clock.cc:648] HybridClock initialized: now 1786544267134451 us; error 29 us; skew 500 ppm
Aug 12 14:17:47 dist-test-slave-5k54 krb5kdc[9154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544267, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:47.137121 9278 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:17:47.137502 9278 webserver.cc:460] Webserver started at http://127.8.85.1:36385/ using document root <none> and password file <none>
I20260812 14:17:47.137679 9278 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:47.137719 9278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:47.137821 9278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:47.138813 9278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "5a847614f7d443b8b86623c028782d98"
format_stamp: "Formatted at 2026-08-12 14:17:47 on dist-test-slave-5k54"
server_key: "67e00a867cc0847003ae5bf6ffdfb67f"
server_key_iv: "c4103e1e62c23c1513953f9cf2c37344"
server_key_version: "encryptionkey@0"
I20260812 14:17:47.139132 9278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "5a847614f7d443b8b86623c028782d98"
format_stamp: "Formatted at 2026-08-12 14:17:47 on dist-test-slave-5k54"
server_key: "67e00a867cc0847003ae5bf6ffdfb67f"
server_key_iv: "c4103e1e62c23c1513953f9cf2c37344"
server_key_version: "encryptionkey@0"
I20260812 14:17:47.140508 9278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:17:47.141255 9295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:47.141430 9278 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:17:47.141530 9278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "5a847614f7d443b8b86623c028782d98"
format_stamp: "Formatted at 2026-08-12 14:17:47 on dist-test-slave-5k54"
server_key: "67e00a867cc0847003ae5bf6ffdfb67f"
server_key_iv: "c4103e1e62c23c1513953f9cf2c37344"
server_key_version: "encryptionkey@0"
I20260812 14:17:47.141608 9278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:47.170145 9278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:47.172081 9278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:47.172246 9278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:47.172582 9278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:17:47.172906 9278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:17:47.172958 9278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:47.173002 9278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:17:47.173026 9278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:47.178790 9278 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:33295
I20260812 14:17:47.178846 9429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:33295 every 8 connection(s)
I20260812 14:17:47.179333 9278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:17:47.182665 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 9278
I20260812 14:17:47.182768 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:17:47.182946 8532 external_mini_cluster.cc:1428] Setting key 4dca20ac56eaae5a298471dcd5f59c55
Aug 12 14:17:47 dist-test-slave-5k54 krb5kdc[9154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544267, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:47.191304 9269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.180191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:57187 (local address 127.8.85.62:39555)
0812 14:17:47.180256 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:47.180258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:47.180483 (+ 225us) server_negotiation.cc:407] Connection header received
0812 14:17:47.181892 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:47.181893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:47.181907 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:47.181969 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:47.182715 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.183416 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.184962 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.185313 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.185384 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.187886 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:47.187888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:47.187889 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:47.187902 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:47.189651 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:47.190153 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:47.190155 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:47.190155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:47.190222 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:47.190500 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:47.190501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:47.190502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:47.190623 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:47.190702 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:47.191090 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 14:17:47.191201 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:17:47.191488 9432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.180158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:39555 (local address 127.8.85.1:57187)
0812 14:17:47.180351 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:47.180405 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:47.181766 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:47.182043 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:47.182047 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:47.182325 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:47.182574 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.182581 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.183552 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.183553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.184865 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.184868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.185394 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.185395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.185520 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.186127 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:47.186135 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:47.187709 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:47.189768 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:47.189776 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:47.189779 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:47.190067 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:47.190295 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:47.190296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:47.190296 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:47.190417 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:47.190693 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:47.190701 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:47.190870 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 14:17:47.191002 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 14:17:47.191198 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":99,"threads_started":1}
I20260812 14:17:47.192000 9430 heartbeater.cc:347] Connected to a master server at 127.8.85.62:39555
I20260812 14:17:47.192142 9430 heartbeater.cc:464] Registering TS with master...
I20260812 14:17:47.192395 9430 heartbeater.cc:511] Master 127.8.85.62:39555 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
I20260812 14:17:47.193023 9205 ts_manager.cc:194] Registered new tserver with Master: 5a847614f7d443b8b86623c028782d98 (127.8.85.1:33295)
I20260812 14:17:47.193687 9205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:57187
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:17:47.210140 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:39555
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:47.292318 9435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:47.292523 9435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:47.292547 9435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:47.293634 9435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:47.293669 9435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:47.293696 9435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:17:47.295176 9435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:39555
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:47.295354 9435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:47.295581 9435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:47.300436 9443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:47.300441 9441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:47.300478 9440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:47.300608 9435 server_base.cc:1034] running on GCE node
I20260812 14:17:47.300793 9435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:47.301050 9435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:47.302210 9435 hybrid_clock.cc:648] HybridClock initialized: now 1786544267302137 us; error 101 us; skew 500 ppm
Aug 12 14:17:47 dist-test-slave-5k54 krb5kdc[9154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544267, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:47.304592 9435 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:17:47.305091 9435 webserver.cc:460] Webserver started at http://127.8.85.2:36345/ using document root <none> and password file <none>
I20260812 14:17:47.305308 9435 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:47.305372 9435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:47.305518 9435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:47.306535 9435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "70ed9f7da75a4f80998a4dbf013c085c"
format_stamp: "Formatted at 2026-08-12 14:17:47 on dist-test-slave-5k54"
server_key: "b73347d372d11e34604b2d5f934fbfbc"
server_key_iv: "699d0c7ab4432623337822d2b9dd1651"
server_key_version: "encryptionkey@0"
I20260812 14:17:47.306900 9435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "70ed9f7da75a4f80998a4dbf013c085c"
format_stamp: "Formatted at 2026-08-12 14:17:47 on dist-test-slave-5k54"
server_key: "b73347d372d11e34604b2d5f934fbfbc"
server_key_iv: "699d0c7ab4432623337822d2b9dd1651"
server_key_version: "encryptionkey@0"
I20260812 14:17:47.308362 9435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:17:47.309118 9451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:47.309286 9435 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:47.309386 9435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "70ed9f7da75a4f80998a4dbf013c085c"
format_stamp: "Formatted at 2026-08-12 14:17:47 on dist-test-slave-5k54"
server_key: "b73347d372d11e34604b2d5f934fbfbc"
server_key_iv: "699d0c7ab4432623337822d2b9dd1651"
server_key_version: "encryptionkey@0"
I20260812 14:17:47.309458 9435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:47.315752 9435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:47.317432 9435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:47.317579 9435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:47.317777 9435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:17:47.318066 9435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:17:47.318116 9435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:47.318156 9435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:17:47.318183 9435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:47.324230 9435 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:37089
I20260812 14:17:47.324266 9579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:37089 every 8 connection(s)
I20260812 14:17:47.324568 9435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:17:47.325688 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 9435
I20260812 14:17:47.325788 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:17:47.325953 8532 external_mini_cluster.cc:1428] Setting key 9d196df958fb341e4a610775b9659596
Aug 12 14:17:47 dist-test-slave-5k54 krb5kdc[9154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544267, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:47.335256 9269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.325335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:58009 (local address 127.8.85.62:39555)
0812 14:17:47.325418 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:47.325420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:47.325598 (+ 178us) server_negotiation.cc:407] Connection header received
0812 14:17:47.326899 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:47.326900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:47.326910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:47.326962 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:47.327910 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.328504 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.329545 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.329755 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.329822 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.332394 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:47.332395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:47.332395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:47.332402 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:47.333784 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:47.334260 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:47.334261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:47.334262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:47.334313 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:47.334593 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:47.334594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:47.334594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:47.334745 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:47.334798 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:47.335124 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 14:17:47.335191 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 14:17:47.335587 9582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.325300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:39555 (local address 127.8.85.2:58009)
0812 14:17:47.325464 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:47.325503 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:47.326762 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:47.327032 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:47.327036 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:47.327506 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:47.327776 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.327782 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.328606 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.328608 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.329459 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.329461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.329839 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.329840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.329971 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.330484 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:47.330492 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:47.332278 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:47.333868 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:47.333876 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:47.333878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:47.334188 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:47.334385 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:47.334386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:47.334386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:47.334529 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:47.334813 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:47.334819 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:47.334983 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 14:17:47.335108 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 14:17:47.335283 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260812 14:17:47.336123 9580 heartbeater.cc:347] Connected to a master server at 127.8.85.62:39555
I20260812 14:17:47.336218 9580 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
I20260812 14:17:47.336408 9580 heartbeater.cc:511] Master 127.8.85.62:39555 requested a full tablet report, sending...
I20260812 14:17:47.336755 9205 ts_manager.cc:194] Registered new tserver with Master: 70ed9f7da75a4f80998a4dbf013c085c (127.8.85.2:37089)
I20260812 14:17:47.337131 9205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:58009
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:17:47.354204 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:39555
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:47.437330 9585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:47.437522 9585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:47.437546 9585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:47.438678 9585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:47.438721 9585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:47.438747 9585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:17:47.440274 9585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:39555
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:47.440502 9585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:47.440733 9585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:47.445667 9590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:47.445693 9591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:47.445676 9594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:47.446015 9585 server_base.cc:1034] running on GCE node
I20260812 14:17:47.446204 9585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:47.446419 9585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:47.447559 9585 hybrid_clock.cc:648] HybridClock initialized: now 1786544267447551 us; error 31 us; skew 500 ppm
Aug 12 14:17:47 dist-test-slave-5k54 krb5kdc[9154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544267, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:47.450063 9585 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:17:47.450481 9585 webserver.cc:460] Webserver started at http://127.8.85.3:39733/ using document root <none> and password file <none>
I20260812 14:17:47.450713 9585 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:47.450778 9585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:47.450932 9585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:47.451992 9585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "b7c1e2d0b054414cafc4015dbfcbfe62"
format_stamp: "Formatted at 2026-08-12 14:17:47 on dist-test-slave-5k54"
server_key: "85f9d095ce607e9871dbe4f16cca8480"
server_key_iv: "8f05d6365530f8119afddddf7c16569c"
server_key_version: "encryptionkey@0"
I20260812 14:17:47.452350 9585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "b7c1e2d0b054414cafc4015dbfcbfe62"
format_stamp: "Formatted at 2026-08-12 14:17:47 on dist-test-slave-5k54"
server_key: "85f9d095ce607e9871dbe4f16cca8480"
server_key_iv: "8f05d6365530f8119afddddf7c16569c"
server_key_version: "encryptionkey@0"
I20260812 14:17:47.453742 9585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:17:47.454413 9601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:47.454577 9585 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:17:47.454663 9585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "b7c1e2d0b054414cafc4015dbfcbfe62"
format_stamp: "Formatted at 2026-08-12 14:17:47 on dist-test-slave-5k54"
server_key: "85f9d095ce607e9871dbe4f16cca8480"
server_key_iv: "8f05d6365530f8119afddddf7c16569c"
server_key_version: "encryptionkey@0"
I20260812 14:17:47.454744 9585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:47.472190 9585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:47.474093 9585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:47.474253 9585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:47.474490 9585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:17:47.474805 9585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:17:47.474856 9585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:47.474891 9585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:17:47.474916 9585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:47.481529 9585 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:46333
I20260812 14:17:47.481575 9732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:46333 every 8 connection(s)
I20260812 14:17:47.481891 9585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:17:47 dist-test-slave-5k54 krb5kdc[9154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544267, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:47.489464 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 9585
I20260812 14:17:47.489566 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:17:47.489755 8532 external_mini_cluster.cc:1428] Setting key afd3fabfe44a54b25bf1cedb46e0aeaa
I20260812 14:17:47.491367 9269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.482569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:46323 (local address 127.8.85.62:39555)
0812 14:17:47.482645 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:47.482648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:47.482858 (+ 210us) server_negotiation.cc:407] Connection header received
0812 14:17:47.483803 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:47.483804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:47.483815 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:47.483854 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:47.484437 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.484978 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.486038 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.486221 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.486271 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.488432 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:47.488433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:47.488433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:47.488440 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:47.490108 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:47.490647 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:47.490647 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:47.490648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:47.490701 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:47.490930 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:47.490931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:47.490931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:47.491027 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:47.491073 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:47.491251 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 14:17:47.491306 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 14:17:47.491628 9738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.482530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:39555 (local address 127.8.85.3:46323)
0812 14:17:47.482710 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:47.482746 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:47.483700 (+ 954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:47.483917 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:47.483921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:47.484168 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:47.484352 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.484356 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.485057 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.485057 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.485955 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.485957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.486287 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.486287 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.486379 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.486808 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:47.486816 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:47.488318 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:47.490240 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:47.490247 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:47.490249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:47.490531 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:47.490782 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:47.490782 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:47.490783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:47.490886 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:47.491063 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:47.491065 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:47.491167 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 14:17:47.491240 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 14:17:47.491351 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260812 14:17:47.492156 9735 heartbeater.cc:347] Connected to a master server at 127.8.85.62:39555
I20260812 14:17:47.492252 9735 heartbeater.cc:464] Registering TS with master...
I20260812 14:17:47.492426 9735 heartbeater.cc:511] Master 127.8.85.62:39555 requested a full tablet report, sending...
I20260812 14:17:47.492772 9205 ts_manager.cc:194] Registered new tserver with Master: b7c1e2d0b054414cafc4015dbfcbfe62 (127.8.85.3:46333)
I20260812 14:17:47.493139 9205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:46323
I20260812 14:17:47.501579 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:17:47.509353 9269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.503582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54602 (local address 127.8.85.62:39555)
0812 14:17:47.503653 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:47.503655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:47.503760 (+ 105us) server_negotiation.cc:407] Connection header received
0812 14:17:47.503881 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:47.503882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:47.503893 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:47.503970 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:47.504528 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.505008 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.505730 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.505923 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.505994 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.507084 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:47.507085 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:47.507086 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:47.507093 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:47.508211 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:47.508559 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:47.508561 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:47.508561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:47.508616 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:47.508802 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:47.508803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:47.508803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:47.508949 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:47.509051 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:47.509163 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 14:17:47.509240 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:17:47.512153 9205 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:17:47.514045 9205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:17:47.521688 9748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.517098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:37089 (local address 127.0.0.1:45606)
0812 14:17:47.517923 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:47.517933 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:47.517983 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:47.518367 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:47.518367 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:47.518369 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:47.518409 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.518410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.519262 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.519263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.520702 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.520704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.521380 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.521381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.521461 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.521475 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 14:17:47.521541 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 14:17:47.521598 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"mutex_wait_us":86,"thread_start_us":112,"threads_started":1}
I20260812 14:17:47.521780 9751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.517309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40832 (local address 127.8.85.3:46333)
0812 14:17:47.517499 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:47.517504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:47.517628 (+ 124us) server_negotiation.cc:407] Connection header received
0812 14:17:47.517768 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:47.517771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:47.517791 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:47.517872 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:47.518099 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.519251 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.520750 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.521446 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.521505 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.521554 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:47.521649 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 14:17:47.521695 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260812 14:17:47.522024 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.517305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47514 (local address 127.8.85.1:33295)
0812 14:17:47.517638 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:47.517641 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:47.517872 (+ 231us) server_negotiation.cc:407] Connection header received
0812 14:17:47.517988 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:47.517991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:47.518010 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:47.518096 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:47.518222 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.519302 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.520610 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.521405 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.521485 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.521545 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:47.521923 (+ 378us) server_negotiation.cc:299] Negotiation successful
0812 14:17:47.521968 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":92,"threads_started":1}
I20260812 14:17:47.522049 9753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.517381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45606 (local address 127.8.85.2:37089)
0812 14:17:47.517577 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:47.517584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:47.518198 (+ 614us) server_negotiation.cc:407] Connection header received
0812 14:17:47.518246 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:47.518248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:47.518268 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:47.518357 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:47.518484 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.519156 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.520787 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.521274 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.521367 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.521434 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:47.521911 (+ 477us) server_negotiation.cc:299] Negotiation successful
0812 14:17:47.521960 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":98,"threads_started":1}
I20260812 14:17:47.521840 9749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.517098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:33295 (local address 127.0.0.1:47514)
0812 14:17:47.517779 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:47.517788 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:47.517854 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:47.518099 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:47.518100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:47.518102 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:47.518159 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.518161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.519469 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.519471 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.520529 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.520532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.521585 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.521587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.521656 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.521660 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:17:47.521712 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 14:17:47.521753 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":117,"threads_started":1}
I20260812 14:17:47.521976 9750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.517093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:46333 (local address 127.0.0.1:40832)
0812 14:17:47.517515 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:47.517561 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:47.517646 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:47.517912 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:47.517916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:47.517918 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:47.517992 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.517994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.519354 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.519355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.520654 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.520658 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.521518 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.521519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.521585 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.521588 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:17:47.521638 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 14:17:47.521714 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":35,"thread_start_us":51,"threads_started":1}
I20260812 14:17:47.522981 9659 tablet_service.cc:1467] Processing CreateTablet for tablet 03e722f4ae9a460ab45dc0010d2e1345 (DEFAULT_TABLE table=test-table [id=638c7c64cef24f6093c03f7bceeb84d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:47.522980 9507 tablet_service.cc:1467] Processing CreateTablet for tablet 03e722f4ae9a460ab45dc0010d2e1345 (DEFAULT_TABLE table=test-table [id=638c7c64cef24f6093c03f7bceeb84d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:47.523312 9507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03e722f4ae9a460ab45dc0010d2e1345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:47.523320 9659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03e722f4ae9a460ab45dc0010d2e1345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:47.524717 9354 tablet_service.cc:1467] Processing CreateTablet for tablet 03e722f4ae9a460ab45dc0010d2e1345 (DEFAULT_TABLE table=test-table [id=638c7c64cef24f6093c03f7bceeb84d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:47.525069 9354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03e722f4ae9a460ab45dc0010d2e1345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:47.526346 9754 tablet_bootstrap.cc:492] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62: Bootstrap starting.
I20260812 14:17:47.527132 9754 tablet_bootstrap.cc:654] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:47.527644 9755 tablet_bootstrap.cc:492] T 03e722f4ae9a460ab45dc0010d2e1345 P 70ed9f7da75a4f80998a4dbf013c085c: Bootstrap starting.
I20260812 14:17:47.527992 9754 log.cc:824] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62: Log is configured to *not* fsync() on all Append() calls
I20260812 14:17:47.528067 9756 tablet_bootstrap.cc:492] T 03e722f4ae9a460ab45dc0010d2e1345 P 5a847614f7d443b8b86623c028782d98: Bootstrap starting.
I20260812 14:17:47.528561 9755 tablet_bootstrap.cc:654] T 03e722f4ae9a460ab45dc0010d2e1345 P 70ed9f7da75a4f80998a4dbf013c085c: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:47.528918 9756 tablet_bootstrap.cc:654] T 03e722f4ae9a460ab45dc0010d2e1345 P 5a847614f7d443b8b86623c028782d98: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:47.529027 9754 tablet_bootstrap.cc:492] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62: No bootstrap required, opened a new log
I20260812 14:17:47.529132 9754 ts_tablet_manager.cc:1397] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:17:47.529181 9755 log.cc:824] T 03e722f4ae9a460ab45dc0010d2e1345 P 70ed9f7da75a4f80998a4dbf013c085c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:17:47.529618 9756 log.cc:824] T 03e722f4ae9a460ab45dc0010d2e1345 P 5a847614f7d443b8b86623c028782d98: Log is configured to *not* fsync() on all Append() calls
I20260812 14:17:47.529994 9755 tablet_bootstrap.cc:492] T 03e722f4ae9a460ab45dc0010d2e1345 P 70ed9f7da75a4f80998a4dbf013c085c: No bootstrap required, opened a new log
I20260812 14:17:47.530072 9755 ts_tablet_manager.cc:1397] T 03e722f4ae9a460ab45dc0010d2e1345 P 70ed9f7da75a4f80998a4dbf013c085c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:17:47.530345 9756 tablet_bootstrap.cc:492] T 03e722f4ae9a460ab45dc0010d2e1345 P 5a847614f7d443b8b86623c028782d98: No bootstrap required, opened a new log
I20260812 14:17:47.530421 9756 ts_tablet_manager.cc:1397] T 03e722f4ae9a460ab45dc0010d2e1345 P 5a847614f7d443b8b86623c028782d98: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:17:47.531210 9754 raft_consensus.cc:348] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a847614f7d443b8b86623c028782d98" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33295 } } peers { permanent_uuid: "70ed9f7da75a4f80998a4dbf013c085c" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37089 } } peers { permanent_uuid: "b7c1e2d0b054414cafc4015dbfcbfe62" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 46333 } }
I20260812 14:17:47.531402 9754 raft_consensus.cc:374] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:47.531484 9754 raft_consensus.cc:729] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7c1e2d0b054414cafc4015dbfcbfe62, State: Initialized, Role: FOLLOWER
I20260812 14:17:47.531621 9754 consensus_queue.cc:260] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a847614f7d443b8b86623c028782d98" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33295 } } peers { permanent_uuid: "70ed9f7da75a4f80998a4dbf013c085c" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37089 } } peers { permanent_uuid: "b7c1e2d0b054414cafc4015dbfcbfe62" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 46333 } }
I20260812 14:17:47.531917 9754 ts_tablet_manager.cc:1428] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:17:47.532119 9735 heartbeater.cc:503] Master 127.8.85.62:39555 was elected leader, sending a full tablet report...
I20260812 14:17:47.532124 9756 raft_consensus.cc:348] T 03e722f4ae9a460ab45dc0010d2e1345 P 5a847614f7d443b8b86623c028782d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a847614f7d443b8b86623c028782d98" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33295 } } peers { permanent_uuid: "70ed9f7da75a4f80998a4dbf013c085c" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37089 } } peers { permanent_uuid: "b7c1e2d0b054414cafc4015dbfcbfe62" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 46333 } }
I20260812 14:17:47.532260 9755 raft_consensus.cc:348] T 03e722f4ae9a460ab45dc0010d2e1345 P 70ed9f7da75a4f80998a4dbf013c085c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a847614f7d443b8b86623c028782d98" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33295 } } peers { permanent_uuid: "70ed9f7da75a4f80998a4dbf013c085c" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37089 } } peers { permanent_uuid: "b7c1e2d0b054414cafc4015dbfcbfe62" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 46333 } }
I20260812 14:17:47.532361 9756 raft_consensus.cc:374] T 03e722f4ae9a460ab45dc0010d2e1345 P 5a847614f7d443b8b86623c028782d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:47.532378 9755 raft_consensus.cc:374] T 03e722f4ae9a460ab45dc0010d2e1345 P 70ed9f7da75a4f80998a4dbf013c085c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:47.532407 9755 raft_consensus.cc:729] T 03e722f4ae9a460ab45dc0010d2e1345 P 70ed9f7da75a4f80998a4dbf013c085c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70ed9f7da75a4f80998a4dbf013c085c, State: Initialized, Role: FOLLOWER
I20260812 14:17:47.532403 9756 raft_consensus.cc:729] T 03e722f4ae9a460ab45dc0010d2e1345 P 5a847614f7d443b8b86623c028782d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a847614f7d443b8b86623c028782d98, State: Initialized, Role: FOLLOWER
I20260812 14:17:47.532533 9756 consensus_queue.cc:260] T 03e722f4ae9a460ab45dc0010d2e1345 P 5a847614f7d443b8b86623c028782d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a847614f7d443b8b86623c028782d98" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33295 } } peers { permanent_uuid: "70ed9f7da75a4f80998a4dbf013c085c" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37089 } } peers { permanent_uuid: "b7c1e2d0b054414cafc4015dbfcbfe62" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 46333 } }
I20260812 14:17:47.532542 9755 consensus_queue.cc:260] T 03e722f4ae9a460ab45dc0010d2e1345 P 70ed9f7da75a4f80998a4dbf013c085c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a847614f7d443b8b86623c028782d98" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33295 } } peers { permanent_uuid: "70ed9f7da75a4f80998a4dbf013c085c" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37089 } } peers { permanent_uuid: "b7c1e2d0b054414cafc4015dbfcbfe62" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 46333 } }
I20260812 14:17:47.532838 9756 ts_tablet_manager.cc:1428] T 03e722f4ae9a460ab45dc0010d2e1345 P 5a847614f7d443b8b86623c028782d98: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 14:17:47.532863 9430 heartbeater.cc:503] Master 127.8.85.62:39555 was elected leader, sending a full tablet report...
I20260812 14:17:47.532894 9755 ts_tablet_manager.cc:1428] T 03e722f4ae9a460ab45dc0010d2e1345 P 70ed9f7da75a4f80998a4dbf013c085c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:17:47.533151 9580 heartbeater.cc:503] Master 127.8.85.62:39555 was elected leader, sending a full tablet report...
W20260812 14:17:47.575093 9581 tablet.cc:2369] T 03e722f4ae9a460ab45dc0010d2e1345 P 70ed9f7da75a4f80998a4dbf013c085c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:17:47.651981 9764 raft_consensus.cc:482] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:17:47.652107 9764 raft_consensus.cc:504] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a847614f7d443b8b86623c028782d98" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33295 } } peers { permanent_uuid: "70ed9f7da75a4f80998a4dbf013c085c" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37089 } } peers { permanent_uuid: "b7c1e2d0b054414cafc4015dbfcbfe62" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 46333 } }
I20260812 14:17:47.652719 9764 leader_election.cc:288] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a847614f7d443b8b86623c028782d98 (127.8.85.1:33295), 70ed9f7da75a4f80998a4dbf013c085c (127.8.85.2:37089)
I20260812 14:17:47.656721 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.653009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:41935 (local address 127.8.85.1:33295)
0812 14:17:47.653073 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:47.653075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:47.653087 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:17:47.653180 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:47.653182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:47.653199 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:47.653280 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:47.653474 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.654216 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.655767 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.656319 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.656417 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.656427 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:47.656611 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 14:17:47.656650 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:17:47.656721 9753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.653092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:41979 (local address 127.8.85.2:37089)
0812 14:17:47.653141 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:47.653143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:47.653234 (+ 91us) server_negotiation.cc:407] Connection header received
0812 14:17:47.653266 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:47.653267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:47.653277 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:47.653335 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:47.653478 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.654246 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.655755 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.656320 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.656399 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.656411 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:47.656611 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 14:17:47.656653 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:17:47.656724 9738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.652930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:33295 (local address 127.8.85.3:41935)
0812 14:17:47.653021 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:47.653032 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:47.653080 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:47.653296 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:47.653297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:47.653299 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:47.653388 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.653390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.654308 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.654309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.655686 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.655688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.656429 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.656431 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.656531 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.656533 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:17:47.656596 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 14:17:47.656637 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":50}
I20260812 14:17:47.656733 9767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.652978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:37089 (local address 127.8.85.3:41979)
0812 14:17:47.653158 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:47.653167 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:47.653216 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:47.653342 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:47.653343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:47.653344 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:47.653408 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.653409 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.654336 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.654337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.655635 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:47.655639 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.656430 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.656431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:47.656519 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.656523 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:17:47.656596 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 14:17:47.656637 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":35,"thread_start_us":121,"threads_started":1}
I20260812 14:17:47.657187 9381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e722f4ae9a460ab45dc0010d2e1345" candidate_uuid: "b7c1e2d0b054414cafc4015dbfcbfe62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a847614f7d443b8b86623c028782d98" is_pre_election: true
I20260812 14:17:47.657279 9530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e722f4ae9a460ab45dc0010d2e1345" candidate_uuid: "b7c1e2d0b054414cafc4015dbfcbfe62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70ed9f7da75a4f80998a4dbf013c085c" is_pre_election: true
I20260812 14:17:47.657459 9530 raft_consensus.cc:2454] T 03e722f4ae9a460ab45dc0010d2e1345 P 70ed9f7da75a4f80998a4dbf013c085c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7c1e2d0b054414cafc4015dbfcbfe62 in term 0.
I20260812 14:17:47.657459 9381 raft_consensus.cc:2454] T 03e722f4ae9a460ab45dc0010d2e1345 P 5a847614f7d443b8b86623c028782d98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7c1e2d0b054414cafc4015dbfcbfe62 in term 0.
I20260812 14:17:47.657783 9605 leader_election.cc:302] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a847614f7d443b8b86623c028782d98, b7c1e2d0b054414cafc4015dbfcbfe62; no voters:
I20260812 14:17:47.657955 9764 raft_consensus.cc:2781] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:17:47.658030 9764 raft_consensus.cc:482] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:17:47.658047 9764 raft_consensus.cc:3037] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:47.658736 9764 raft_consensus.cc:504] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a847614f7d443b8b86623c028782d98" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33295 } } peers { permanent_uuid: "70ed9f7da75a4f80998a4dbf013c085c" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37089 } } peers { permanent_uuid: "b7c1e2d0b054414cafc4015dbfcbfe62" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 46333 } }
I20260812 14:17:47.658905 9764 leader_election.cc:288] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [CANDIDATE]: Term 1 election: Requested vote from peers 5a847614f7d443b8b86623c028782d98 (127.8.85.1:33295), 70ed9f7da75a4f80998a4dbf013c085c (127.8.85.2:37089)
I20260812 14:17:47.659053 9381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e722f4ae9a460ab45dc0010d2e1345" candidate_uuid: "b7c1e2d0b054414cafc4015dbfcbfe62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a847614f7d443b8b86623c028782d98"
I20260812 14:17:47.659092 9530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03e722f4ae9a460ab45dc0010d2e1345" candidate_uuid: "b7c1e2d0b054414cafc4015dbfcbfe62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70ed9f7da75a4f80998a4dbf013c085c"
I20260812 14:17:47.659149 9381 raft_consensus.cc:3037] T 03e722f4ae9a460ab45dc0010d2e1345 P 5a847614f7d443b8b86623c028782d98 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:47.659155 9530 raft_consensus.cc:3037] T 03e722f4ae9a460ab45dc0010d2e1345 P 70ed9f7da75a4f80998a4dbf013c085c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:47.659935 9530 raft_consensus.cc:2454] T 03e722f4ae9a460ab45dc0010d2e1345 P 70ed9f7da75a4f80998a4dbf013c085c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7c1e2d0b054414cafc4015dbfcbfe62 in term 1.
I20260812 14:17:47.659943 9381 raft_consensus.cc:2454] T 03e722f4ae9a460ab45dc0010d2e1345 P 5a847614f7d443b8b86623c028782d98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7c1e2d0b054414cafc4015dbfcbfe62 in term 1.
I20260812 14:17:47.660137 9603 leader_election.cc:302] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70ed9f7da75a4f80998a4dbf013c085c, b7c1e2d0b054414cafc4015dbfcbfe62; no voters:
I20260812 14:17:47.660234 9764 raft_consensus.cc:2781] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:17:47.660413 9764 raft_consensus.cc:686] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [term 1 LEADER]: Becoming Leader. State: Replica: b7c1e2d0b054414cafc4015dbfcbfe62, State: Running, Role: LEADER
I20260812 14:17:47.660526 9764 consensus_queue.cc:237] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a847614f7d443b8b86623c028782d98" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33295 } } peers { permanent_uuid: "70ed9f7da75a4f80998a4dbf013c085c" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37089 } } peers { permanent_uuid: "b7c1e2d0b054414cafc4015dbfcbfe62" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 46333 } }
I20260812 14:17:47.661485 9204 catalog_manager.cc:5495] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7c1e2d0b054414cafc4015dbfcbfe62 (127.8.85.3). New cstate: current_term: 1 leader_uuid: "b7c1e2d0b054414cafc4015dbfcbfe62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a847614f7d443b8b86623c028782d98" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70ed9f7da75a4f80998a4dbf013c085c" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7c1e2d0b054414cafc4015dbfcbfe62" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 46333 } health_report { overall_health: HEALTHY } } }
W20260812 14:17:47.680140 9431 tablet.cc:2369] T 03e722f4ae9a460ab45dc0010d2e1345 P 5a847614f7d443b8b86623c028782d98: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 14:17:47 dist-test-slave-5k54 krb5kdc[9154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.1@KRBTEST.COM
I20260812 14:17:47.705637 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.698374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47524 (local address 127.8.85.1:33295)
0812 14:17:47.698458 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:47.698460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:47.698566 (+ 106us) server_negotiation.cc:407] Connection header received
0812 14:17:47.698637 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:47.698638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:47.698649 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:47.698712 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:47.699290 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.699816 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.700517 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.700709 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.700780 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.703177 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:47.703189 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:47.703189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:47.703197 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:47.704656 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:47.704988 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:47.704991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:47.704991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:47.705059 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:47.705218 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:47.705219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:47.705219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:47.705354 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:47.705465 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:47.705514 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 14:17:47.705572 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 14:17:47 dist-test-slave-5k54 krb5kdc[9154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544267, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 14:17:47 dist-test-slave-5k54 krb5kdc[9154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544267, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.85.1@KRBTEST.COM
I20260812 14:17:47.729375 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.722760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47538 (local address 127.8.85.1:33295)
0812 14:17:47.722833 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:47.722835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:47.722914 (+ 79us) server_negotiation.cc:407] Connection header received
0812 14:17:47.722936 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:47.722937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:47.722946 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:47.722987 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:47.723556 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.724035 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.724740 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.724965 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.725019 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.727161 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:47.727162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:47.727163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:47.727169 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:47.728375 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:47.728666 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:47.728667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:47.728667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:47.728715 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:47.728893 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:47.728893 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:47.728893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:47.728978 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:47.729048 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:47.729201 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 14:17:47.729281 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 14:17:47.732415 9737 tablet.cc:2369] T 03e722f4ae9a460ab45dc0010d2e1345 P b7c1e2d0b054414cafc4015dbfcbfe62: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 14:17:47 dist-test-slave-5k54 krb5kdc[9154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 14:17:47 dist-test-slave-5k54 krb5kdc[9154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.1@KRBTEST.COM
I20260812 14:17:47.752460 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:47.745756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47548 (local address 127.8.85.1:33295)
0812 14:17:47.745818 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:47.745820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:47.745911 (+ 91us) server_negotiation.cc:407] Connection header received
0812 14:17:47.745991 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:47.745993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:47.746003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:47.746069 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:47.746582 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.747055 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.747836 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:47.748027 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:47.748120 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:47.750150 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:47.750152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:47.750152 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:47.750159 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:47.751315 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:47.751719 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:47.751720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:47.751720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:47.751789 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:47.751990 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:47.751990 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:47.751991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:47.752086 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:47.752185 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:47.752282 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 14:17:47.752358 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260812 14:17:47.753260 9327 tablet_service.cc:2908] Rejecting scan request for tablet 03e722f4ae9a460ab45dc0010d2e1345: Uninitialized: clean time has not yet been initialized
I20260812 14:17:47.754498 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 9278
I20260812 14:17:47.766378 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 9435
I20260812 14:17:47.771883 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 9585
I20260812 14:17:47.779349 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 9170
2026-08-12T14:17:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2180 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:17:47 dist-test-slave-5k54 krb5kdc[9785](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:17:47 dist-test-slave-5k54 krb5kdc[9785](info): set up 2 sockets
Aug 12 14:17:47 dist-test-slave-5k54 krb5kdc[9785](info): commencing operation
krb5kdc: starting...
W20260812 14:17:48.916435 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:17:49 dist-test-slave-5k54 krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:17:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:17:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:17:49.040593 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:42813
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:45349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:42813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:49.124466 9800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:49.124684 9800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:49.124708 9800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:49.125927 9800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:17:49.125972 9800 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 14:17:49.125986 9800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:49.126000 9800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:49.126013 9800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:17:49.126025 9800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:17:49.127666 9800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:45349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:42813
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:42813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:49.127924 9800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:49.128171 9800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:49.133562 9809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:49.133598 9805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:49.133668 9800 server_base.cc:1034] running on GCE node
W20260812 14:17:49.133630 9806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:49.134063 9800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:49.134325 9800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:49.135507 9800 hybrid_clock.cc:648] HybridClock initialized: now 1786544269135482 us; error 37 us; skew 500 ppm
Aug 12 14:17:49 dist-test-slave-5k54 krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544269, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:49.148047 9800 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:17:49.148463 9800 webserver.cc:460] Webserver started at http://127.8.85.62:32957/ using document root <none> and password file <none>
I20260812 14:17:49.148669 9800 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:49.148710 9800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:49.148803 9800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:49.149871 9800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "cfa4a857ca184b9f86b25e8c7a65b92e"
format_stamp: "Formatted at 2026-08-12 14:17:49 on dist-test-slave-5k54"
server_key: "17753966c7671048c8bd97108db948ce"
server_key_iv: "397519725aacfdcbb1f398b2e622bb47"
server_key_version: "encryptionkey@0"
I20260812 14:17:49.150265 9800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "cfa4a857ca184b9f86b25e8c7a65b92e"
format_stamp: "Formatted at 2026-08-12 14:17:49 on dist-test-slave-5k54"
server_key: "17753966c7671048c8bd97108db948ce"
server_key_iv: "397519725aacfdcbb1f398b2e622bb47"
server_key_version: "encryptionkey@0"
I20260812 14:17:49.151618 9800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:17:49.152359 9819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:49.152527 9800 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:17:49.152632 9800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "cfa4a857ca184b9f86b25e8c7a65b92e"
format_stamp: "Formatted at 2026-08-12 14:17:49 on dist-test-slave-5k54"
server_key: "17753966c7671048c8bd97108db948ce"
server_key_iv: "397519725aacfdcbb1f398b2e622bb47"
server_key_version: "encryptionkey@0"
I20260812 14:17:49.152706 9800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:49.163790 9800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:49.166030 9800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:49.166185 9800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:49.171066 9800 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:42813
I20260812 14:17:49.171120 9888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:42813 every 8 connection(s)
I20260812 14:17:49.171494 9800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:17:49.172128 9889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:49.174448 9889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e: Bootstrap starting.
I20260812 14:17:49.175068 9889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:49.175360 9889 log.cc:824] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:17:49.175838 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 9800
I20260812 14:17:49.175976 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:17:49.176139 8532 external_mini_cluster.cc:1428] Setting key 3d5f134ced4d3a62e297bd3aa79362e4
I20260812 14:17:49.176152 9889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e: No bootstrap required, opened a new log
I20260812 14:17:49.179203 9889 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa4a857ca184b9f86b25e8c7a65b92e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42813 } }
I20260812 14:17:49.179383 9889 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:49.179445 9889 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfa4a857ca184b9f86b25e8c7a65b92e, State: Initialized, Role: FOLLOWER
I20260812 14:17:49.179562 9889 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa4a857ca184b9f86b25e8c7a65b92e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42813 } }
I20260812 14:17:49.179637 9889 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:17:49.179663 9889 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:17:49.179704 9889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:49.180327 9889 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa4a857ca184b9f86b25e8c7a65b92e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42813 } }
I20260812 14:17:49.180429 9889 leader_election.cc:302] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfa4a857ca184b9f86b25e8c7a65b92e; no voters:
I20260812 14:17:49.180612 9889 leader_election.cc:288] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:17:49.180673 9895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:17:49.180822 9895 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [term 1 LEADER]: Becoming Leader. State: Replica: cfa4a857ca184b9f86b25e8c7a65b92e, State: Running, Role: LEADER
I20260812 14:17:49.180889 9889 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:17:49.180913 9895 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa4a857ca184b9f86b25e8c7a65b92e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42813 } }
I20260812 14:17:49.181255 9895 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfa4a857ca184b9f86b25e8c7a65b92e. Latest consensus state: current_term: 1 leader_uuid: "cfa4a857ca184b9f86b25e8c7a65b92e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa4a857ca184b9f86b25e8c7a65b92e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42813 } } }
I20260812 14:17:49.181332 9895 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [sys.catalog]: This master's current role is: LEADER
I20260812 14:17:49.181264 9896 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfa4a857ca184b9f86b25e8c7a65b92e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa4a857ca184b9f86b25e8c7a65b92e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42813 } } }
I20260812 14:17:49.181383 9896 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e [sys.catalog]: This master's current role is: LEADER
I20260812 14:17:49.181855 9901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:17:49.182595 9901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 14:17:49 dist-test-slave-5k54 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:49.184787 9901 catalog_manager.cc:1347] Generated new cluster ID: a3c0af0d14544099919bfc681882562a
I20260812 14:17:49.184844 9901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:17:49.187724 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.176631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51182 (local address 127.8.85.62:42813)
0812 14:17:49.176826 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:49.176836 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:49.176871 (+ 35us) server_negotiation.cc:407] Connection header received
0812 14:17:49.177602 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:49.177612 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:49.177676 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:49.177820 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:49.178435 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.179880 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.180759 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.181753 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.181872 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.184633 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:49.184638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:49.184638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:49.184653 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:49.186183 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.186564 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.186567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.186567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.186655 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.186871 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.186872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.186872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.187105 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:49.187243 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:49.187364 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 14:17:49.187527 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260812 14:17:49.191265 9901 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:17:49.191910 9901 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:17:49.196077 9901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e: Generated new TSK 0
I20260812 14:17:49.196226 9901 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:17:49.230085 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:42813
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:45349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:49.335196 9918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:49.335426 9918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:49.335495 9918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:49.336681 9918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:49.336721 9918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:49.336741 9918 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:17:49.336759 9918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:17:49.338182 9918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:45349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.85.62:42813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:49.338419 9918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:49.338640 9918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:49.343887 9927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:49.343946 9924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:49.344099 9918 server_base.cc:1034] running on GCE node
W20260812 14:17:49.344115 9929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:49.344370 9918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:49.344657 9918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:49.345817 9918 hybrid_clock.cc:648] HybridClock initialized: now 1786544269345806 us; error 30 us; skew 500 ppm
Aug 12 14:17:49 dist-test-slave-5k54 krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544269, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:49.348651 9918 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:17:49.349176 9918 webserver.cc:460] Webserver started at http://127.8.85.1:39777/ using document root <none> and password file <none>
I20260812 14:17:49.349434 9918 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:49.349506 9918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:49.349658 9918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:49.350771 9918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "a9a90b5383524e099750ac0423fb22ff"
format_stamp: "Formatted at 2026-08-12 14:17:49 on dist-test-slave-5k54"
server_key: "401baface8b0a7b54cabfcfea535c9b8"
server_key_iv: "a1d24a424b017b068706473398d80589"
server_key_version: "encryptionkey@0"
I20260812 14:17:49.351197 9918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "a9a90b5383524e099750ac0423fb22ff"
format_stamp: "Formatted at 2026-08-12 14:17:49 on dist-test-slave-5k54"
server_key: "401baface8b0a7b54cabfcfea535c9b8"
server_key_iv: "a1d24a424b017b068706473398d80589"
server_key_version: "encryptionkey@0"
I20260812 14:17:49.352759 9918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:17:49.353541 9937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:49.353711 9918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:17:49.353820 9918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "a9a90b5383524e099750ac0423fb22ff"
format_stamp: "Formatted at 2026-08-12 14:17:49 on dist-test-slave-5k54"
server_key: "401baface8b0a7b54cabfcfea535c9b8"
server_key_iv: "a1d24a424b017b068706473398d80589"
server_key_version: "encryptionkey@0"
I20260812 14:17:49.353933 9918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:49.374810 9918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:49.376924 9918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:49.377097 9918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:49.377785 9918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:17:49.377866 9918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:49.377902 9918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:17:49.377924 9918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 14:17:49 dist-test-slave-5k54 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544269, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:49.386175 9918 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:46649
I20260812 14:17:49.386673 9918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:17:49.387199 10074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:46649 every 8 connection(s)
I20260812 14:17:49.388625 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.378342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:37585 (local address 127.8.85.62:42813)
0812 14:17:49.378414 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:49.378417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:49.378673 (+ 256us) server_negotiation.cc:407] Connection header received
0812 14:17:49.379199 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:49.379200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:49.379215 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:49.379300 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:49.380054 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.380633 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.382014 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.382354 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.382452 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.385563 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:49.385565 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:49.385565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:49.385574 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:49.387413 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.387850 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.387852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.387852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.387913 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.388115 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.388116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.388116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.388223 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:49.388274 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:49.388490 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 14:17:49.388560 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 14:17:49.388785 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.378323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:42813 (local address 127.8.85.1:37585)
0812 14:17:49.378521 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:49.378566 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:49.379097 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:49.379462 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:49.379466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:49.379726 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:49.379938 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.379942 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.380772 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.380773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.381914 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.381917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.382463 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.382465 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.382635 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.383247 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:49.383257 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:49.385457 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:49.387523 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:49.387529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:49.387531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:49.387760 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:49.387971 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:49.387971 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:49.387972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:49.388062 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:49.388264 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:49.388267 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:49.388384 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 14:17:49.388466 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 14:17:49.388589 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":121,"threads_started":1}
I20260812 14:17:49.389273 10075 heartbeater.cc:347] Connected to a master server at 127.8.85.62:42813
I20260812 14:17:49.389359 10075 heartbeater.cc:464] Registering TS with master...
I20260812 14:17:49.389550 10075 heartbeater.cc:511] Master 127.8.85.62:42813 requested a full tablet report, sending...
I20260812 14:17:49.390122 9837 ts_manager.cc:194] Registered new tserver with Master: a9a90b5383524e099750ac0423fb22ff (127.8.85.1:46649)
I20260812 14:17:49.390918 9837 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:37585
I20260812 14:17:49.396312 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 9918
I20260812 14:17:49.396431 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:17:49.396557 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.390457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:52887 (local address 127.8.85.62:42813)
0812 14:17:49.390501 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:49.390503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:49.390629 (+ 126us) server_negotiation.cc:407] Connection header received
0812 14:17:49.390673 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:49.390674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:49.390683 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:49.390746 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:49.391090 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.391755 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.392567 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.392788 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.392881 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.393844 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:49.393855 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:49.393855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:49.393862 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:49.395318 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.395729 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.395731 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.395732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.395789 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.396019 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.396021 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.396021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.396134 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:49.396211 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:49.396380 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 14:17:49.396463 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:17:49.396575 10081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.390398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:42813 (local address 127.8.85.1:52887)
0812 14:17:49.390560 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:49.390575 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:49.390626 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:49.390750 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:49.390751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:49.390908 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:49.390986 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.390987 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.391855 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.391857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.392480 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.392482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.392883 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.392884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.392963 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.393415 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:49.393421 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:49.393733 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:49.395454 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:49.395455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:49.395456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:49.395638 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:49.395882 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:49.395883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:49.395884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:49.395952 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:49.396203 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:49.396204 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:49.396263 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 14:17:49.396359 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 14:17:49.396464 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":73,"threads_started":1}
I20260812 14:17:49.396639 8532 external_mini_cluster.cc:1428] Setting key 6a318586c29a8d9f6681d6d48f1fe392
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:17:49.423067 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:42813
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:45349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:49.507071 10085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:49.507282 10085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:49.507306 10085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:49.508498 10085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:49.508543 10085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:49.508564 10085 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:17:49.508582 10085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:17:49.509923 10085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:45349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.85.62:42813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:49.510099 10085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:49.510269 10085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:49.515098 10092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:49.515214 10085 server_base.cc:1034] running on GCE node
W20260812 14:17:49.515111 10096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:49.515122 10094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:49.515511 10085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:49.515784 10085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:49.516947 10085 hybrid_clock.cc:648] HybridClock initialized: now 1786544269516929 us; error 29 us; skew 500 ppm
Aug 12 14:17:49 dist-test-slave-5k54 krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544269, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:49.519639 10085 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:17:49.520160 10085 webserver.cc:460] Webserver started at http://127.8.85.2:37665/ using document root <none> and password file <none>
I20260812 14:17:49.520393 10085 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:49.520458 10085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:49.520612 10085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:49.521675 10085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "93ea6c9a214746258ef6005d8da97e86"
format_stamp: "Formatted at 2026-08-12 14:17:49 on dist-test-slave-5k54"
server_key: "d2e895aaed3e7b0f4da71769cff4808b"
server_key_iv: "70336feb730023ae79f04727c6e70d2d"
server_key_version: "encryptionkey@0"
I20260812 14:17:49.522058 10085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "93ea6c9a214746258ef6005d8da97e86"
format_stamp: "Formatted at 2026-08-12 14:17:49 on dist-test-slave-5k54"
server_key: "d2e895aaed3e7b0f4da71769cff4808b"
server_key_iv: "70336feb730023ae79f04727c6e70d2d"
server_key_version: "encryptionkey@0"
I20260812 14:17:49.523466 10085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:17:49.524269 10105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:49.524456 10085 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:17:49.524555 10085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "93ea6c9a214746258ef6005d8da97e86"
format_stamp: "Formatted at 2026-08-12 14:17:49 on dist-test-slave-5k54"
server_key: "d2e895aaed3e7b0f4da71769cff4808b"
server_key_iv: "70336feb730023ae79f04727c6e70d2d"
server_key_version: "encryptionkey@0"
I20260812 14:17:49.524632 10085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:49.532948 10085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:49.535228 10085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:49.535391 10085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:49.535959 10085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:17:49.536019 10085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:49.536063 10085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:17:49.536091 10085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:49.543330 10085 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:38845
I20260812 14:17:49.543458 10239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:38845 every 8 connection(s)
I20260812 14:17:49.543731 10085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:17:49 dist-test-slave-5k54 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544269, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:49.547500 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.536540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:47977 (local address 127.8.85.62:42813)
0812 14:17:49.536601 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:49.536603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:49.536859 (+ 256us) server_negotiation.cc:407] Connection header received
0812 14:17:49.537500 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:49.537502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:49.537517 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:49.537584 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:49.538532 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.539520 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.541260 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.541542 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.541602 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.544432 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:49.544434 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:49.544435 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:49.544444 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:49.545979 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.546434 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.546436 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.546436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.546501 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.546769 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.546770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.546771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.546896 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:49.546977 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:49.547273 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 14:17:49.547362 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:17:49.547672 10114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.536515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:42813 (local address 127.8.85.2:47977)
0812 14:17:49.536732 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:49.536769 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:49.537335 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:49.537681 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:49.537685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:49.538095 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:49.538434 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.538442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.539620 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.539622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.541141 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.541144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.541646 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.541647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.541806 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.542408 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:49.542419 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:49.544322 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:49.546102 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:49.546109 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:49.546111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:49.546349 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:49.546573 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:49.546574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:49.546574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:49.546696 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:49.546968 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:49.546975 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:49.547122 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 14:17:49.547234 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 14:17:49.547405 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":141,"threads_started":1}
I20260812 14:17:49.548125 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 10085
I20260812 14:17:49.548221 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:17:49.548264 10240 heartbeater.cc:347] Connected to a master server at 127.8.85.62:42813
I20260812 14:17:49.548362 10240 heartbeater.cc:464] Registering TS with master...
I20260812 14:17:49.548403 8532 external_mini_cluster.cc:1428] Setting key f8c2bf80c7145125678d3d43e5deaaa1
I20260812 14:17:49.548556 10240 heartbeater.cc:511] Master 127.8.85.62:42813 requested a full tablet report, sending...
I20260812 14:17:49.548966 9836 ts_manager.cc:194] Registered new tserver with Master: 93ea6c9a214746258ef6005d8da97e86 (127.8.85.2:38845)
I20260812 14:17:49.549441 9836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:47977
I20260812 14:17:49.555510 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.549438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:47949 (local address 127.8.85.62:42813)
0812 14:17:49.549476 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:49.549479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:49.549699 (+ 220us) server_negotiation.cc:407] Connection header received
0812 14:17:49.549777 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:49.549778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:49.549789 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:49.549837 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:49.550205 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.550736 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.551530 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.551753 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.551828 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.552833 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:49.552835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:49.552835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:49.552842 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:49.554278 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.554630 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.554632 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.554632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.554692 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.554902 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.554903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.554904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.555061 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:49.555144 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:49.555303 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 14:17:49.555384 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:17:49.555521 10247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.549393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:42813 (local address 127.8.85.2:47949)
0812 14:17:49.549632 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:49.549642 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:49.549714 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:49.549853 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:49.549854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:49.550053 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:49.550113 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.550114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.550833 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.550834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.551412 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.551413 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.551841 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.551842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.551932 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.552417 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:49.552422 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:49.552718 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:49.554370 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:49.554371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:49.554372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:49.554553 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:49.554768 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:49.554769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:49.554770 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:49.554832 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:49.555136 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:49.555137 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:49.555186 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 14:17:49.555276 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 14:17:49.555384 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:17:49.575968 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:42813
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:45349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:49.659920 10250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:49.660118 10250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:49.660141 10250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:49.661368 10250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:49.661417 10250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:49.661437 10250 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:17:49.661478 10250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:17:49.662940 10250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:45349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.85.62:42813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:49.663262 10250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:49.663578 10250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:49.668872 10257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:49.668905 10256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:49.668853 10259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:49.669310 10250 server_base.cc:1034] running on GCE node
I20260812 14:17:49.669540 10250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:49.669771 10250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:49.670928 10250 hybrid_clock.cc:648] HybridClock initialized: now 1786544269670916 us; error 32 us; skew 500 ppm
Aug 12 14:17:49 dist-test-slave-5k54 krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544269, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:49.673493 10250 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:17:49.673871 10250 webserver.cc:460] Webserver started at http://127.8.85.3:43935/ using document root <none> and password file <none>
I20260812 14:17:49.674052 10250 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:49.674093 10250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:49.674181 10250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:49.675176 10250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "21bcc1fbd9d84815a27e6ee574118aab"
format_stamp: "Formatted at 2026-08-12 14:17:49 on dist-test-slave-5k54"
server_key: "7195bf29ce4645af1568553d2d26d73c"
server_key_iv: "9f084ce7b56599c3d3d7eccc9f5ae95d"
server_key_version: "encryptionkey@0"
I20260812 14:17:49.675559 10250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "21bcc1fbd9d84815a27e6ee574118aab"
format_stamp: "Formatted at 2026-08-12 14:17:49 on dist-test-slave-5k54"
server_key: "7195bf29ce4645af1568553d2d26d73c"
server_key_iv: "9f084ce7b56599c3d3d7eccc9f5ae95d"
server_key_version: "encryptionkey@0"
I20260812 14:17:49.676823 10250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:17:49.677495 10267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:49.677659 10250 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:17:49.677728 10250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "21bcc1fbd9d84815a27e6ee574118aab"
format_stamp: "Formatted at 2026-08-12 14:17:49 on dist-test-slave-5k54"
server_key: "7195bf29ce4645af1568553d2d26d73c"
server_key_iv: "9f084ce7b56599c3d3d7eccc9f5ae95d"
server_key_version: "encryptionkey@0"
I20260812 14:17:49.677780 10250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:49.686041 10250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:49.687879 10250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:49.687990 10250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:49.688417 10250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:17:49.688458 10250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:49.688500 10250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:17:49.688527 10250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 14:17:49 dist-test-slave-5k54 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544269, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:49.695739 10250 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:34039
I20260812 14:17:49.696086 10402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:34039 every 8 connection(s)
I20260812 14:17:49.696307 10250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:17:49.698262 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.688860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:51703 (local address 127.8.85.62:42813)
0812 14:17:49.688901 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:49.688904 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:49.689126 (+ 222us) server_negotiation.cc:407] Connection header received
0812 14:17:49.689698 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:49.689699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:49.689712 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:49.689791 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:49.690593 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.691398 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.693070 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.693288 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.693369 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.695524 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:49.695525 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:49.695525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:49.695533 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:49.697161 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.697519 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.697520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.697520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.697574 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.697796 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.697796 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.697796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.697892 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:49.697939 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:49.698139 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 14:17:49.698199 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:17:49.698477 10276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.688854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:42813 (local address 127.8.85.3:51703)
0812 14:17:49.689014 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:49.689048 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:49.689573 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:49.689896 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:49.689899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:49.690233 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:49.690505 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.690512 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.691522 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.691523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.692975 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.692979 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.693402 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.693403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.693518 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.694000 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:49.694008 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:49.695381 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:49.697231 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:49.697236 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:49.697238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:49.697450 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:49.697659 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:49.697660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:49.697660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:49.697754 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:49.697941 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:49.697945 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:49.698049 (+ 104us) client_negotiation.cc:769] Sending connection context
0812 14:17:49.698128 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 14:17:49.698239 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
I20260812 14:17:49.698966 10403 heartbeater.cc:347] Connected to a master server at 127.8.85.62:42813
I20260812 14:17:49.699048 10403 heartbeater.cc:464] Registering TS with master...
I20260812 14:17:49.699224 10403 heartbeater.cc:511] Master 127.8.85.62:42813 requested a full tablet report, sending...
I20260812 14:17:49.699630 9836 ts_manager.cc:194] Registered new tserver with Master: 21bcc1fbd9d84815a27e6ee574118aab (127.8.85.3:34039)
I20260812 14:17:49.700026 9836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:51703
I20260812 14:17:49.700839 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 10250
I20260812 14:17:49.700937 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:17:49.701114 8532 external_mini_cluster.cc:1428] Setting key 5bbf9503e46c6f853f427f17070cfd16
I20260812 14:17:49.701975 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:17:49.705781 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.700043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:43891 (local address 127.8.85.62:42813)
0812 14:17:49.700082 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:49.700084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:49.700152 (+ 68us) server_negotiation.cc:407] Connection header received
0812 14:17:49.700230 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:49.700231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:49.700241 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:49.700303 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:49.700660 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.701250 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.702010 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.702267 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.702333 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.703173 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:49.703175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:49.703175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:49.703183 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:49.704719 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.705126 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.705127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.705127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.705176 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.705365 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.705366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.705366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.705456 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:49.705501 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:49.705670 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 14:17:49.705725 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:17:49.705926 10409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.699990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:42813 (local address 127.8.85.3:43891)
0812 14:17:49.700098 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:49.700107 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:49.700164 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:49.700305 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:49.700306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:49.700519 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:49.700586 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.700588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.701325 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.701326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.701951 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.701953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.702339 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.702340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.702395 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.702780 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:49.702784 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:49.703088 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:49.704810 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:49.704811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:49.704812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:49.705050 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:49.705232 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:49.705233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:49.705234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:49.705303 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:49.705511 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:49.705512 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:49.705558 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 14:17:49.705639 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 14:17:49.705745 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":63,"threads_started":1}
Aug 12 14:17:49 dist-test-slave-5k54 krb5kdc[9785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544269, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 14:17:49 dist-test-slave-5k54 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544269, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:49.726332 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.719620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51188 (local address 127.8.85.62:42813)
0812 14:17:49.719661 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:49.719663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:49.719672 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:17:49.719735 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:49.719736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:49.719745 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:49.719800 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:49.720366 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.720867 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.721606 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.721800 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.721870 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.724122 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:49.724123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:49.724124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:49.724131 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:49.725493 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.725779 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.725780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.725780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.725827 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:49.725972 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:49.725972 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:49.725972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:49.726059 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:49.726130 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:49.726218 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 14:17:49.726272 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:17:49.727960 9836 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:17:49.729635 9836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:17:49.736133 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.731968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:46649 (local address 127.0.0.1:47278)
0812 14:17:49.732088 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:49.732130 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:49.732192 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:49.732554 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:49.732558 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:49.732560 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:49.732645 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.732647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.733653 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.733654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.734892 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.734895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.735853 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.735854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.735944 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.735964 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 14:17:49.736039 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 14:17:49.736080 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":30,"thread_start_us":74,"threads_started":1}
I20260812 14:17:49.736208 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.732086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47278 (local address 127.8.85.1:46649)
0812 14:17:49.732347 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:49.732354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:49.732366 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:17:49.732409 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:49.732411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:49.732427 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:49.732518 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:49.732748 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.733573 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.734985 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.735771 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.735849 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.735914 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:49.736064 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 14:17:49.736111 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":96,"threads_started":1}
I20260812 14:17:49.736496 10422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.732185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:34039 (local address 127.0.0.1:60518)
0812 14:17:49.732320 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:49.732329 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:49.732368 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:49.732879 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:49.732879 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:49.732881 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:49.732924 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.732925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.733793 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.733794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.734978 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.734981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.736244 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.736245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.736332 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.736337 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:17:49.736392 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 14:17:49.736436 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260812 14:17:49.736567 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.732285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60518 (local address 127.8.85.3:34039)
0812 14:17:49.732712 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:49.732718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:49.732729 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:17:49.732770 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:49.732774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:49.732790 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:49.732882 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:49.732990 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.733698 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.735054 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.735770 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.735849 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.735913 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:49.736430 (+ 517us) server_negotiation.cc:299] Negotiation successful
0812 14:17:49.736474 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":107,"threads_started":1}
I20260812 14:17:49.737197 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.732362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:38845 (local address 127.0.0.1:37384)
0812 14:17:49.732645 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:49.732654 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:49.732746 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:49.733882 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:49.733883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:49.733885 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:49.733931 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.733933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.735133 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.735134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.735900 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:49.735902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.736966 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.736967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:49.737025 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.737029 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:17:49.737086 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 14:17:49.737122 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":79,"threads_started":1}
I20260812 14:17:49.737254 10425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:49.732547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37384 (local address 127.8.85.2:38845)
0812 14:17:49.733708 (+ 1161us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:49.733711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:49.733719 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:17:49.733772 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:49.733775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:49.733790 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:49.733874 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:49.733996 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.735075 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.735982 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:49.736805 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:49.736877 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:49.736931 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:49.737115 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 14:17:49.737155 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1131,"thread_start_us":86,"threads_started":1}
I20260812 14:17:49.737221 9995 tablet_service.cc:1467] Processing CreateTablet for tablet d58250b1e25e442e9264c44e7eba702c (DEFAULT_TABLE table=test-table [id=0e85198de9a047e1b12e94a1f4d741d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:49.737401 10328 tablet_service.cc:1467] Processing CreateTablet for tablet d58250b1e25e442e9264c44e7eba702c (DEFAULT_TABLE table=test-table [id=0e85198de9a047e1b12e94a1f4d741d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:49.737594 9995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d58250b1e25e442e9264c44e7eba702c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:49.737673 10328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d58250b1e25e442e9264c44e7eba702c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:49.740414 10426 tablet_bootstrap.cc:492] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab: Bootstrap starting.
I20260812 14:17:49.740394 10161 tablet_service.cc:1467] Processing CreateTablet for tablet d58250b1e25e442e9264c44e7eba702c (DEFAULT_TABLE table=test-table [id=0e85198de9a047e1b12e94a1f4d741d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:49.740715 10161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d58250b1e25e442e9264c44e7eba702c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:49.741293 10426 tablet_bootstrap.cc:654] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:49.741537 10426 log.cc:824] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab: Log is configured to *not* fsync() on all Append() calls
I20260812 14:17:49.741930 10427 tablet_bootstrap.cc:492] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff: Bootstrap starting.
I20260812 14:17:49.742321 10426 tablet_bootstrap.cc:492] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab: No bootstrap required, opened a new log
I20260812 14:17:49.742401 10426 ts_tablet_manager.cc:1397] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:17:49.742518 10427 tablet_bootstrap.cc:654] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:49.742821 10427 log.cc:824] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff: Log is configured to *not* fsync() on all Append() calls
I20260812 14:17:49.743662 10427 tablet_bootstrap.cc:492] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff: No bootstrap required, opened a new log
I20260812 14:17:49.743762 10427 ts_tablet_manager.cc:1397] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:17:49.743997 10431 tablet_bootstrap.cc:492] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86: Bootstrap starting.
I20260812 14:17:49.744623 10431 tablet_bootstrap.cc:654] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:49.744628 10426 raft_consensus.cc:348] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } }
I20260812 14:17:49.744822 10426 raft_consensus.cc:374] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:49.744879 10426 raft_consensus.cc:729] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21bcc1fbd9d84815a27e6ee574118aab, State: Initialized, Role: FOLLOWER
I20260812 14:17:49.744985 10431 log.cc:824] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86: Log is configured to *not* fsync() on all Append() calls
I20260812 14:17:49.745002 10426 consensus_queue.cc:260] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } }
I20260812 14:17:49.745275 10426 ts_tablet_manager.cc:1428] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:17:49.745402 10403 heartbeater.cc:503] Master 127.8.85.62:42813 was elected leader, sending a full tablet report...
I20260812 14:17:49.745671 10427 raft_consensus.cc:348] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } }
I20260812 14:17:49.745851 10427 raft_consensus.cc:374] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:49.745903 10427 raft_consensus.cc:729] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a90b5383524e099750ac0423fb22ff, State: Initialized, Role: FOLLOWER
I20260812 14:17:49.746022 10427 consensus_queue.cc:260] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } }
I20260812 14:17:49.746099 10431 tablet_bootstrap.cc:492] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86: No bootstrap required, opened a new log
I20260812 14:17:49.746156 10431 ts_tablet_manager.cc:1397] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:17:49.746299 10427 ts_tablet_manager.cc:1428] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 14:17:49.746376 10075 heartbeater.cc:503] Master 127.8.85.62:42813 was elected leader, sending a full tablet report...
I20260812 14:17:49.747915 10431 raft_consensus.cc:348] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } }
I20260812 14:17:49.748035 10431 raft_consensus.cc:374] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:49.748056 10431 raft_consensus.cc:729] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93ea6c9a214746258ef6005d8da97e86, State: Initialized, Role: FOLLOWER
I20260812 14:17:49.748200 10431 consensus_queue.cc:260] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } }
I20260812 14:17:49.748467 10431 ts_tablet_manager.cc:1428] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:17:49.748521 10240 heartbeater.cc:503] Master 127.8.85.62:42813 was elected leader, sending a full tablet report...
W20260812 14:17:49.794309 10241 tablet.cc:2369] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:17:49.887537 10076 tablet.cc:2369] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:17:49.946969 10404 tablet.cc:2369] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:17:50.008279 10436 raft_consensus.cc:482] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:17:50.008420 10436 raft_consensus.cc:504] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } }
I20260812 14:17:50.008828 10436 leader_election.cc:288] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21bcc1fbd9d84815a27e6ee574118aab (127.8.85.3:34039), 93ea6c9a214746258ef6005d8da97e86 (127.8.85.2:38845)
I20260812 14:17:50.013099 10425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:50.009368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:54319 (local address 127.8.85.2:38845)
0812 14:17:50.009414 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:50.009417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:50.009466 (+ 49us) server_negotiation.cc:407] Connection header received
0812 14:17:50.009516 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:50.009517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:50.009529 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:50.009577 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:50.009767 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.010548 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.012111 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.012660 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.012729 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:50.012739 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:50.012967 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 14:17:50.013023 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:17:50.013113 10440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:50.009282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:38845 (local address 127.8.85.1:54319)
0812 14:17:50.009398 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:50.009408 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:50.009465 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:50.009595 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:50.009596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:50.009597 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:50.009690 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:50.009693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.010641 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.010642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:50.012022 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:50.012025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.012783 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.012783 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:50.012873 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:50.012881 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 14:17:50.012946 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 14:17:50.013002 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":106,"thread_start_us":63,"threads_started":1}
I20260812 14:17:50.013170 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:50.009169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:58699 (local address 127.8.85.3:34039)
0812 14:17:50.009235 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:50.009238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:50.009316 (+ 78us) server_negotiation.cc:407] Connection header received
0812 14:17:50.009464 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:50.009466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:50.009482 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:50.009552 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:50.009763 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.010555 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.012104 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.012654 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.012729 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:50.012739 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:50.013052 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 14:17:50.013093 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:17:50.013345 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:50.009044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:34039 (local address 127.8.85.1:58699)
0812 14:17:50.009226 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:50.009240 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:50.009338 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:50.009565 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:50.009566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:50.009568 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:50.009690 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:50.009694 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.010640 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.010641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:50.012022 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:50.012025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.012780 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.012781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:50.012873 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:50.012880 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 14:17:50.012946 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 14:17:50.013002 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":74,"thread_start_us":106,"threads_started":1}
I20260812 14:17:50.013674 10349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d58250b1e25e442e9264c44e7eba702c" candidate_uuid: "a9a90b5383524e099750ac0423fb22ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21bcc1fbd9d84815a27e6ee574118aab" is_pre_election: true
I20260812 14:17:50.013679 10185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d58250b1e25e442e9264c44e7eba702c" candidate_uuid: "a9a90b5383524e099750ac0423fb22ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93ea6c9a214746258ef6005d8da97e86" is_pre_election: true
I20260812 14:17:50.013837 10349 raft_consensus.cc:2454] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a90b5383524e099750ac0423fb22ff in term 0.
I20260812 14:17:50.013844 10185 raft_consensus.cc:2454] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a90b5383524e099750ac0423fb22ff in term 0.
I20260812 14:17:50.014097 9939 leader_election.cc:302] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93ea6c9a214746258ef6005d8da97e86, a9a90b5383524e099750ac0423fb22ff; no voters:
I20260812 14:17:50.014225 10436 raft_consensus.cc:2781] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:17:50.014303 10436 raft_consensus.cc:482] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:17:50.014328 10436 raft_consensus.cc:3037] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:50.015115 10436 raft_consensus.cc:504] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } }
I20260812 14:17:50.015252 10436 leader_election.cc:288] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [CANDIDATE]: Term 1 election: Requested vote from peers 21bcc1fbd9d84815a27e6ee574118aab (127.8.85.3:34039), 93ea6c9a214746258ef6005d8da97e86 (127.8.85.2:38845)
I20260812 14:17:50.015463 10185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d58250b1e25e442e9264c44e7eba702c" candidate_uuid: "a9a90b5383524e099750ac0423fb22ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93ea6c9a214746258ef6005d8da97e86"
I20260812 14:17:50.015478 10349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d58250b1e25e442e9264c44e7eba702c" candidate_uuid: "a9a90b5383524e099750ac0423fb22ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21bcc1fbd9d84815a27e6ee574118aab"
I20260812 14:17:50.015553 10185 raft_consensus.cc:3037] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:50.015553 10349 raft_consensus.cc:3037] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:50.016479 10185 raft_consensus.cc:2454] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a90b5383524e099750ac0423fb22ff in term 1.
I20260812 14:17:50.016479 10349 raft_consensus.cc:2454] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a90b5383524e099750ac0423fb22ff in term 1.
I20260812 14:17:50.016652 9939 leader_election.cc:302] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93ea6c9a214746258ef6005d8da97e86, a9a90b5383524e099750ac0423fb22ff; no voters:
I20260812 14:17:50.016743 10436 raft_consensus.cc:2781] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:17:50.016850 10436 raft_consensus.cc:686] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [term 1 LEADER]: Becoming Leader. State: Replica: a9a90b5383524e099750ac0423fb22ff, State: Running, Role: LEADER
I20260812 14:17:50.016952 10436 consensus_queue.cc:237] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } }
I20260812 14:17:50.017803 9836 catalog_manager.cc:5495] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff reported cstate change: term changed from 0 to 1, leader changed from <none> to a9a90b5383524e099750ac0423fb22ff (127.8.85.1). New cstate: current_term: 1 leader_uuid: "a9a90b5383524e099750ac0423fb22ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } health_report { overall_health: UNKNOWN } } }
I20260812 14:17:50.089062 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:50.086587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47292 (local address 127.8.85.1:46649)
0812 14:17:50.086643 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:50.086649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:50.086660 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:17:50.086745 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:50.086746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:50.086768 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:50.086824 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:17:50.086980 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.087461 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.088261 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.088457 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.088515 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:50.088726 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:17:50.088843 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:17:50.088909 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:50.088981 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 14:17:50.089001 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:17:50.091099 10185 raft_consensus.cc:1261] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86 [term 1 FOLLOWER]: Refusing update from remote peer a9a90b5383524e099750ac0423fb22ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:50.091099 10349 raft_consensus.cc:1261] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab [term 1 FOLLOWER]: Refusing update from remote peer a9a90b5383524e099750ac0423fb22ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:50.091524 10436 consensus_queue.cc:1035] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:17:50.093699 10436 consensus_queue.cc:1035] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:17:50.095599 10445 mvcc.cc:205] Tried to move back new op lower bound from 7317685330289823744 to 7317685329990144000. Current Snapshot: MvccSnapshot[applied={T|T < 7317685330289823744 or (T in {7317685330289823744})}]
I20260812 14:17:50.098963 9837 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:17:50.099076 9837 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:17:50.100034 9837 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e: Sending DeleteTablet for 3 replicas of tablet d58250b1e25e442e9264c44e7eba702c
I20260812 14:17:50.100548 10161 tablet_service.cc:1514] Processing DeleteTablet for tablet d58250b1e25e442e9264c44e7eba702c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:50 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:37384
I20260812 14:17:50.100775 9995 tablet_service.cc:1514] Processing DeleteTablet for tablet d58250b1e25e442e9264c44e7eba702c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:50 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:47278
I20260812 14:17:50.101008 10459 tablet_replica.cc:330] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff: stopping tablet replica
I20260812 14:17:50.101128 10459 raft_consensus.cc:2229] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:17:50.101249 10459 raft_consensus.cc:2258] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:50.101671 10459 ts_tablet_manager.cc:1905] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:50.101970 10328 tablet_service.cc:1514] Processing DeleteTablet for tablet d58250b1e25e442e9264c44e7eba702c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:50 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:60518
I20260812 14:17:50.102221 10460 tablet_replica.cc:330] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab: stopping tablet replica
I20260812 14:17:50.102361 10460 raft_consensus.cc:2229] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:17:50.102425 9837 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 14:17:50.102523 10460 raft_consensus.cc:2258] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 14:17:50.102598 9837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:17:50.103075 10457 tablet_replica.cc:330] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86: stopping tablet replica
I20260812 14:17:50.103226 10457 raft_consensus.cc:2229] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:17:50.103412 10457 raft_consensus.cc:2258] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:50.103533 10459 ts_tablet_manager.cc:1918] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:17:50.103693 10459 log.cc:1196] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/wal/wals/d58250b1e25e442e9264c44e7eba702c
I20260812 14:17:50.103967 10457 ts_tablet_manager.cc:1905] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:50.104107 10459 ts_tablet_manager.cc:1939] T d58250b1e25e442e9264c44e7eba702c P a9a90b5383524e099750ac0423fb22ff: Deleting consensus metadata
I20260812 14:17:50.104382 9995 tablet_service.cc:1467] Processing CreateTablet for tablet 6d836ee9a2bf439aae0844312fb12323 (DEFAULT_TABLE table=test-table [id=2d655b0137d24d41bb132b852942de5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:50.104487 9995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d836ee9a2bf439aae0844312fb12323. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:50.104606 9821 catalog_manager.cc:4841] TS a9a90b5383524e099750ac0423fb22ff (127.8.85.1:46649): tablet d58250b1e25e442e9264c44e7eba702c (table test-table [id=0e85198de9a047e1b12e94a1f4d741d8]) successfully deleted
I20260812 14:17:50.105685 10427 tablet_bootstrap.cc:492] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff: Bootstrap starting.
I20260812 14:17:50.105727 10161 tablet_service.cc:1467] Processing CreateTablet for tablet 6d836ee9a2bf439aae0844312fb12323 (DEFAULT_TABLE table=test-table [id=2d655b0137d24d41bb132b852942de5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:50.106108 10427 tablet_bootstrap.cc:654] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:50.106208 10328 tablet_service.cc:1467] Processing CreateTablet for tablet 6d836ee9a2bf439aae0844312fb12323 (DEFAULT_TABLE table=test-table [id=2d655b0137d24d41bb132b852942de5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:50.106298 10328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d836ee9a2bf439aae0844312fb12323. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:50.106482 10161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d836ee9a2bf439aae0844312fb12323. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:50.107682 10426 tablet_bootstrap.cc:492] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab: Bootstrap starting.
I20260812 14:17:50.107707 10431 tablet_bootstrap.cc:492] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86: Bootstrap starting.
I20260812 14:17:50.108160 10431 tablet_bootstrap.cc:654] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:50.108271 10426 tablet_bootstrap.cc:654] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:50.108609 10460 ts_tablet_manager.cc:1905] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:50.108683 10427 tablet_bootstrap.cc:492] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff: No bootstrap required, opened a new log
I20260812 14:17:50.108737 10427 ts_tablet_manager.cc:1397] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260812 14:17:50.108883 10427 raft_consensus.cc:348] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.108975 10427 raft_consensus.cc:374] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:50.108996 10427 raft_consensus.cc:729] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a90b5383524e099750ac0423fb22ff, State: Initialized, Role: FOLLOWER
I20260812 14:17:50.109037 10427 consensus_queue.cc:260] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.109122 10427 ts_tablet_manager.cc:1428] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:50.109313 10457 ts_tablet_manager.cc:1918] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:17:50.109385 10457 log.cc:1196] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/d58250b1e25e442e9264c44e7eba702c
I20260812 14:17:50.109683 10457 ts_tablet_manager.cc:1939] T d58250b1e25e442e9264c44e7eba702c P 93ea6c9a214746258ef6005d8da97e86: Deleting consensus metadata
I20260812 14:17:50.110080 9821 catalog_manager.cc:4841] TS 93ea6c9a214746258ef6005d8da97e86 (127.8.85.2:38845): tablet d58250b1e25e442e9264c44e7eba702c (table test-table [id=0e85198de9a047e1b12e94a1f4d741d8]) successfully deleted
I20260812 14:17:50.110306 10460 ts_tablet_manager.cc:1918] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:17:50.110435 10460 log.cc:1196] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/d58250b1e25e442e9264c44e7eba702c
I20260812 14:17:50.110492 10426 tablet_bootstrap.cc:492] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab: No bootstrap required, opened a new log
I20260812 14:17:50.110517 10431 tablet_bootstrap.cc:492] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86: No bootstrap required, opened a new log
I20260812 14:17:50.110543 10426 ts_tablet_manager.cc:1397] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 14:17:50.110563 10431 ts_tablet_manager.cc:1397] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 14:17:50.110684 10426 raft_consensus.cc:348] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.110749 10426 raft_consensus.cc:374] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:50.110769 10426 raft_consensus.cc:729] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21bcc1fbd9d84815a27e6ee574118aab, State: Initialized, Role: FOLLOWER
I20260812 14:17:50.110785 10460 ts_tablet_manager.cc:1939] T d58250b1e25e442e9264c44e7eba702c P 21bcc1fbd9d84815a27e6ee574118aab: Deleting consensus metadata
I20260812 14:17:50.110810 10426 consensus_queue.cc:260] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.110888 10426 ts_tablet_manager.cc:1428] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:50.110917 10431 raft_consensus.cc:348] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.110993 10431 raft_consensus.cc:374] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:50.111013 10431 raft_consensus.cc:729] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93ea6c9a214746258ef6005d8da97e86, State: Initialized, Role: FOLLOWER
I20260812 14:17:50.111060 10431 consensus_queue.cc:260] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.111166 10431 ts_tablet_manager.cc:1428] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:17:50.111407 9821 catalog_manager.cc:4841] TS 21bcc1fbd9d84815a27e6ee574118aab (127.8.85.3:34039): tablet d58250b1e25e442e9264c44e7eba702c (table test-table [id=0e85198de9a047e1b12e94a1f4d741d8]) successfully deleted
I20260812 14:17:50.422032 10453 raft_consensus.cc:482] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:17:50.422145 10453 raft_consensus.cc:504] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.422307 10453 leader_election.cc:288] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93ea6c9a214746258ef6005d8da97e86 (127.8.85.2:38845), 21bcc1fbd9d84815a27e6ee574118aab (127.8.85.3:34039)
I20260812 14:17:50.422588 10185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d836ee9a2bf439aae0844312fb12323" candidate_uuid: "a9a90b5383524e099750ac0423fb22ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93ea6c9a214746258ef6005d8da97e86" is_pre_election: true
I20260812 14:17:50.422704 10185 raft_consensus.cc:2454] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a90b5383524e099750ac0423fb22ff in term 0.
I20260812 14:17:50.422637 10349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d836ee9a2bf439aae0844312fb12323" candidate_uuid: "a9a90b5383524e099750ac0423fb22ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21bcc1fbd9d84815a27e6ee574118aab" is_pre_election: true
I20260812 14:17:50.422761 10349 raft_consensus.cc:2454] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a90b5383524e099750ac0423fb22ff in term 0.
I20260812 14:17:50.422909 9939 leader_election.cc:302] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93ea6c9a214746258ef6005d8da97e86, a9a90b5383524e099750ac0423fb22ff; no voters:
I20260812 14:17:50.423015 10453 raft_consensus.cc:2781] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:17:50.423062 10453 raft_consensus.cc:482] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:17:50.423091 10453 raft_consensus.cc:3037] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:50.423930 10453 raft_consensus.cc:504] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.424067 10453 leader_election.cc:288] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [CANDIDATE]: Term 1 election: Requested vote from peers 93ea6c9a214746258ef6005d8da97e86 (127.8.85.2:38845), 21bcc1fbd9d84815a27e6ee574118aab (127.8.85.3:34039)
I20260812 14:17:50.424263 10185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d836ee9a2bf439aae0844312fb12323" candidate_uuid: "a9a90b5383524e099750ac0423fb22ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93ea6c9a214746258ef6005d8da97e86"
I20260812 14:17:50.424255 10349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d836ee9a2bf439aae0844312fb12323" candidate_uuid: "a9a90b5383524e099750ac0423fb22ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21bcc1fbd9d84815a27e6ee574118aab"
I20260812 14:17:50.424350 10185 raft_consensus.cc:3037] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:50.424348 10349 raft_consensus.cc:3037] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:50.425035 10349 raft_consensus.cc:2454] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a90b5383524e099750ac0423fb22ff in term 1.
I20260812 14:17:50.425134 10185 raft_consensus.cc:2454] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a90b5383524e099750ac0423fb22ff in term 1.
I20260812 14:17:50.425220 9939 leader_election.cc:302] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21bcc1fbd9d84815a27e6ee574118aab, a9a90b5383524e099750ac0423fb22ff; no voters:
I20260812 14:17:50.425326 10453 raft_consensus.cc:2781] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:17:50.425386 10453 raft_consensus.cc:686] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [term 1 LEADER]: Becoming Leader. State: Replica: a9a90b5383524e099750ac0423fb22ff, State: Running, Role: LEADER
I20260812 14:17:50.425444 10453 consensus_queue.cc:237] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.426048 9837 catalog_manager.cc:5495] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff reported cstate change: term changed from 0 to 1, leader changed from <none> to a9a90b5383524e099750ac0423fb22ff (127.8.85.1). New cstate: current_term: 1 leader_uuid: "a9a90b5383524e099750ac0423fb22ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } health_report { overall_health: UNKNOWN } } }
Aug 12 14:17:50 dist-test-slave-5k54 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544269, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:50.478871 10470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:50.472081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:42813 (local address 127.0.0.1:51204)
0812 14:17:50.472302 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:50.472316 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:50.472395 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:50.472577 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:50.472578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:50.472926 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:50.472971 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:50.472973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.473651 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.473652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:50.474196 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:50.474197 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.474554 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.474555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:50.474605 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:50.475104 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:50.475109 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:50.476561 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:50.477943 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:50.477950 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:50.477952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:50.478149 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:50.478322 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:50.478323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:50.478323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:50.478396 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:50.478597 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:50.478603 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:50.478647 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 14:17:50.478699 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 14:17:50.478788 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":131,"threads_started":1}
I20260812 14:17:50.479028 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:50.472164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.8.85.62:42813)
0812 14:17:50.472333 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:50.472336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:50.472372 (+ 36us) server_negotiation.cc:407] Connection header received
0812 14:17:50.472466 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:50.472467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:50.472479 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:50.472557 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:50.473061 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.473566 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.474275 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.474473 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.474553 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:50.476641 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:50.476643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:50.476643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:50.476649 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:50.477866 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:50.478213 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:50.478213 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:50.478214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:50.478267 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:50.478446 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:50.478447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:50.478447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:50.478547 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:50.478597 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:50.478736 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 14:17:50.478863 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260812 14:17:50.481328 9837 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:51204:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 14:17:50.481683 9837 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:17:50.483146 10161 tablet_service.cc:1467] Processing CreateTablet for tablet c278fa5e9f034b10b4f0304bf651779f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d208beda4be4d76ab5a132f0105f8d1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:17:50.483155 9995 tablet_service.cc:1467] Processing CreateTablet for tablet c278fa5e9f034b10b4f0304bf651779f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d208beda4be4d76ab5a132f0105f8d1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:17:50.483147 10328 tablet_service.cc:1467] Processing CreateTablet for tablet c278fa5e9f034b10b4f0304bf651779f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7d208beda4be4d76ab5a132f0105f8d1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:17:50.483376 9995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c278fa5e9f034b10b4f0304bf651779f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:50.483381 10328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c278fa5e9f034b10b4f0304bf651779f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:50.483372 10161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c278fa5e9f034b10b4f0304bf651779f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:50.485952 10426 tablet_bootstrap.cc:492] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab: Bootstrap starting.
I20260812 14:17:50.485954 10431 tablet_bootstrap.cc:492] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86: Bootstrap starting.
I20260812 14:17:50.486352 10431 tablet_bootstrap.cc:654] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:50.486392 10426 tablet_bootstrap.cc:654] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:50.486708 10427 tablet_bootstrap.cc:492] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff: Bootstrap starting.
I20260812 14:17:50.487013 10426 tablet_bootstrap.cc:492] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab: No bootstrap required, opened a new log
I20260812 14:17:50.487048 10426 ts_tablet_manager.cc:1397] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:17:50.487080 10427 tablet_bootstrap.cc:654] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:50.487088 10431 tablet_bootstrap.cc:492] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86: No bootstrap required, opened a new log
I20260812 14:17:50.487128 10431 ts_tablet_manager.cc:1397] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:17:50.487160 10426 raft_consensus.cc:348] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.487210 10426 raft_consensus.cc:374] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:50.487222 10426 raft_consensus.cc:729] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21bcc1fbd9d84815a27e6ee574118aab, State: Initialized, Role: FOLLOWER
I20260812 14:17:50.487279 10426 consensus_queue.cc:260] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.487278 10431 raft_consensus.cc:348] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.487356 10431 raft_consensus.cc:374] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:50.487385 10426 ts_tablet_manager.cc:1428] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:50.487386 10431 raft_consensus.cc:729] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93ea6c9a214746258ef6005d8da97e86, State: Initialized, Role: FOLLOWER
I20260812 14:17:50.487479 10431 consensus_queue.cc:260] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.487485 10433 tablet_replica.cc:439] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } }
I20260812 14:17:50.487565 10433 tablet_replica.cc:442] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:17:50.487597 10431 ts_tablet_manager.cc:1428] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:17:50.487643 10438 tablet_replica.cc:439] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } }
I20260812 14:17:50.487749 10438 tablet_replica.cc:442] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:17:50.488338 10427 tablet_bootstrap.cc:492] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff: No bootstrap required, opened a new log
I20260812 14:17:50.488380 10427 ts_tablet_manager.cc:1397] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 14:17:50.488511 10427 raft_consensus.cc:348] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.488588 10427 raft_consensus.cc:374] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:50.488608 10427 raft_consensus.cc:729] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a90b5383524e099750ac0423fb22ff, State: Initialized, Role: FOLLOWER
I20260812 14:17:50.488677 10427 consensus_queue.cc:260] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.488786 10427 ts_tablet_manager.cc:1428] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:50.488818 10453 tablet_replica.cc:439] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } }
I20260812 14:17:50.488919 10453 tablet_replica.cc:442] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:17:50.571013 10453 raft_consensus.cc:482] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:17:50.571108 10453 raft_consensus.cc:504] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.571266 10453 leader_election.cc:288] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93ea6c9a214746258ef6005d8da97e86 (127.8.85.2:38845), 21bcc1fbd9d84815a27e6ee574118aab (127.8.85.3:34039)
I20260812 14:17:50.571484 10185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c278fa5e9f034b10b4f0304bf651779f" candidate_uuid: "a9a90b5383524e099750ac0423fb22ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93ea6c9a214746258ef6005d8da97e86" is_pre_election: true
I20260812 14:17:50.571585 10185 raft_consensus.cc:2454] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a90b5383524e099750ac0423fb22ff in term 0.
I20260812 14:17:50.571535 10349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c278fa5e9f034b10b4f0304bf651779f" candidate_uuid: "a9a90b5383524e099750ac0423fb22ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21bcc1fbd9d84815a27e6ee574118aab" is_pre_election: true
I20260812 14:17:50.571650 10349 raft_consensus.cc:2454] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9a90b5383524e099750ac0423fb22ff in term 0.
I20260812 14:17:50.571761 9939 leader_election.cc:302] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93ea6c9a214746258ef6005d8da97e86, a9a90b5383524e099750ac0423fb22ff; no voters:
I20260812 14:17:50.571870 10453 raft_consensus.cc:2781] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:17:50.571944 10453 raft_consensus.cc:482] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:17:50.571979 10453 raft_consensus.cc:3037] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:50.572700 10453 raft_consensus.cc:504] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.572837 10453 leader_election.cc:288] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [CANDIDATE]: Term 1 election: Requested vote from peers 93ea6c9a214746258ef6005d8da97e86 (127.8.85.2:38845), 21bcc1fbd9d84815a27e6ee574118aab (127.8.85.3:34039)
I20260812 14:17:50.573014 10185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c278fa5e9f034b10b4f0304bf651779f" candidate_uuid: "a9a90b5383524e099750ac0423fb22ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93ea6c9a214746258ef6005d8da97e86"
I20260812 14:17:50.573089 10185 raft_consensus.cc:3037] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:50.573066 10349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c278fa5e9f034b10b4f0304bf651779f" candidate_uuid: "a9a90b5383524e099750ac0423fb22ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21bcc1fbd9d84815a27e6ee574118aab"
I20260812 14:17:50.573160 10349 raft_consensus.cc:3037] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:50.573701 10185 raft_consensus.cc:2454] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a90b5383524e099750ac0423fb22ff in term 1.
I20260812 14:17:50.573771 10349 raft_consensus.cc:2454] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a90b5383524e099750ac0423fb22ff in term 1.
I20260812 14:17:50.573920 9939 leader_election.cc:302] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93ea6c9a214746258ef6005d8da97e86, a9a90b5383524e099750ac0423fb22ff; no voters:
I20260812 14:17:50.574044 10453 raft_consensus.cc:2781] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:17:50.574122 10453 raft_consensus.cc:686] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [term 1 LEADER]: Becoming Leader. State: Replica: a9a90b5383524e099750ac0423fb22ff, State: Running, Role: LEADER
I20260812 14:17:50.574180 10453 consensus_queue.cc:237] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } }
I20260812 14:17:50.574445 10436 tablet_replica.cc:439] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff: TxnStatusTablet state changed. Reason: New leader a9a90b5383524e099750ac0423fb22ff. Latest consensus state: current_term: 1 leader_uuid: "a9a90b5383524e099750ac0423fb22ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } }
I20260812 14:17:50.574534 10436 tablet_replica.cc:442] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:17:50.574709 10478 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 14:17:50.574838 10478 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 14:17:50.574774 9836 catalog_manager.cc:5495] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff reported cstate change: term changed from 0 to 1, leader changed from <none> to a9a90b5383524e099750ac0423fb22ff (127.8.85.1). New cstate: current_term: 1 leader_uuid: "a9a90b5383524e099750ac0423fb22ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } health_report { overall_health: UNKNOWN } } }
I20260812 14:17:50.581475 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:50.578645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47300 (local address 127.8.85.1:46649)
0812 14:17:50.578692 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:50.578695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:50.578704 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:17:50.578772 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:50.578773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:50.578784 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:50.578854 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:17:50.578948 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.579685 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.580590 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.580876 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.580965 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:50.581158 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:17:50.581193 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:17:50.581248 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:50.581386 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 14:17:50.581409 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:17:50.581496 10470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:50.578571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:46649 (local address 127.0.0.1:47300)
0812 14:17:50.578643 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:50.578656 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:50.578707 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:50.578859 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:50.578859 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:50.578860 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:17:50.578906 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:50.578909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.579757 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.579757 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:50.580462 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:50.580464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.580999 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.581000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:50.581070 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:50.581086 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:17:50.581266 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:17:50.581275 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 14:17:50.581376 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 14:17:50.581441 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 14:17:50.582533 10349 raft_consensus.cc:1261] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab [term 1 FOLLOWER]: Refusing update from remote peer a9a90b5383524e099750ac0423fb22ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:50.582551 10185 raft_consensus.cc:1261] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86 [term 1 FOLLOWER]: Refusing update from remote peer a9a90b5383524e099750ac0423fb22ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:50.582751 10436 consensus_queue.cc:1035] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:17:50.582844 10453 consensus_queue.cc:1035] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:17:50.583288 10438 tablet_replica.cc:439] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86: TxnStatusTablet state changed. Reason: New leader a9a90b5383524e099750ac0423fb22ff. Latest consensus state: current_term: 1 leader_uuid: "a9a90b5383524e099750ac0423fb22ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } }
I20260812 14:17:50.583381 10438 tablet_replica.cc:442] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:17:50.583408 10433 tablet_replica.cc:439] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab: TxnStatusTablet state changed. Reason: New leader a9a90b5383524e099750ac0423fb22ff. Latest consensus state: current_term: 1 leader_uuid: "a9a90b5383524e099750ac0423fb22ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } }
I20260812 14:17:50.583745 10433 tablet_replica.cc:442] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:17:50.584173 10436 tablet_replica.cc:439] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9a90b5383524e099750ac0423fb22ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } }
I20260812 14:17:50.584246 10436 tablet_replica.cc:442] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:17:50.584460 10438 tablet_replica.cc:439] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9a90b5383524e099750ac0423fb22ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } }
I20260812 14:17:50.584537 10438 tablet_replica.cc:442] T c278fa5e9f034b10b4f0304bf651779f P 93ea6c9a214746258ef6005d8da97e86: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:17:50.585094 10436 tablet_replica.cc:439] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9a90b5383524e099750ac0423fb22ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } }
I20260812 14:17:50.585182 10436 tablet_replica.cc:442] T c278fa5e9f034b10b4f0304bf651779f P a9a90b5383524e099750ac0423fb22ff: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:17:50.585727 10433 tablet_replica.cc:439] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9a90b5383524e099750ac0423fb22ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a90b5383524e099750ac0423fb22ff" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 46649 } } peers { permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 } } peers { permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 } } }
I20260812 14:17:50.585809 10433 tablet_replica.cc:442] T c278fa5e9f034b10b4f0304bf651779f P 21bcc1fbd9d84815a27e6ee574118aab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:17:50.591622 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:50.589111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:35747 (local address 127.8.85.1:46649)
0812 14:17:50.589150 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:50.589152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:50.589288 (+ 136us) server_negotiation.cc:407] Connection header received
0812 14:17:50.589353 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:50.589353 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:50.589362 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:50.589430 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:17:50.589544 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.589992 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.590858 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.591053 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.591130 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:50.591301 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:17:50.591335 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:17:50.591400 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:50.591491 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 14:17:50.591525 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":21}
I20260812 14:17:50.591630 10485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:50.589047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:46649 (local address 127.8.85.1:35747)
0812 14:17:50.589216 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:50.589226 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:50.589288 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:50.589441 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:50.589442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:50.589443 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:17:50.589486 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:50.589488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.590073 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.590074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:50.590773 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:50.590775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.591132 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.591133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:50.591180 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:50.591201 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:17:50.591405 (+ 204us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:17:50.591407 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:17:50.591486 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 14:17:50.591546 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":2,"thread_start_us":83,"threads_started":1}
I20260812 14:17:50.593565 10185 raft_consensus.cc:1261] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86 [term 1 FOLLOWER]: Refusing update from remote peer a9a90b5383524e099750ac0423fb22ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:50.593601 10349 raft_consensus.cc:1261] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab [term 1 FOLLOWER]: Refusing update from remote peer a9a90b5383524e099750ac0423fb22ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:50.593767 10436 consensus_queue.cc:1035] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "93ea6c9a214746258ef6005d8da97e86" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:17:50.593879 10453 consensus_queue.cc:1035] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "21bcc1fbd9d84815a27e6ee574118aab" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:17:50.595074 10449 mvcc.cc:205] Tried to move back new op lower bound from 7317685332349628416 to 7317685331663110144. Current Snapshot: MvccSnapshot[applied={T|T < 7317685332349628416}]
I20260812 14:17:50.596351 10446 mvcc.cc:205] Tried to move back new op lower bound from 7317685332349628416 to 7317685331663110144. Current Snapshot: MvccSnapshot[applied={T|T < 7317685332349628416}]
I20260812 14:17:50.604880 9835 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:17:50.604995 9835 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:17:50.605660 9835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfa4a857ca184b9f86b25e8c7a65b92e: Sending DeleteTablet for 3 replicas of tablet 6d836ee9a2bf439aae0844312fb12323
I20260812 14:17:50.605930 10161 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d836ee9a2bf439aae0844312fb12323 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:50 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:37384
I20260812 14:17:50.605949 10328 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d836ee9a2bf439aae0844312fb12323 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:50 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:60518
I20260812 14:17:50.606022 10457 tablet_replica.cc:330] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86: stopping tablet replica
I20260812 14:17:50.606029 10460 tablet_replica.cc:330] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab: stopping tablet replica
I20260812 14:17:50.606065 10457 raft_consensus.cc:2229] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:17:50.606070 10460 raft_consensus.cc:2229] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:17:50.606096 10457 raft_consensus.cc:2258] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:50.606101 10460 raft_consensus.cc:2258] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:50.606410 10460 ts_tablet_manager.cc:1905] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:50.606410 10457 ts_tablet_manager.cc:1905] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:50.608053 9995 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d836ee9a2bf439aae0844312fb12323 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:50 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:47278
I20260812 14:17:50.608283 10490 tablet_replica.cc:330] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff: stopping tablet replica
I20260812 14:17:50.608346 10457 ts_tablet_manager.cc:1918] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:17:50.608356 10460 ts_tablet_manager.cc:1918] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:17:50.608402 10457 log.cc:1196] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/6d836ee9a2bf439aae0844312fb12323
I20260812 14:17:50.608407 10460 log.cc:1196] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/6d836ee9a2bf439aae0844312fb12323
I20260812 14:17:50.608398 10490 raft_consensus.cc:2229] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:17:50.608464 10490 raft_consensus.cc:2258] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:50.608664 10457 ts_tablet_manager.cc:1939] T 6d836ee9a2bf439aae0844312fb12323 P 93ea6c9a214746258ef6005d8da97e86: Deleting consensus metadata
I20260812 14:17:50.608671 10460 ts_tablet_manager.cc:1939] T 6d836ee9a2bf439aae0844312fb12323 P 21bcc1fbd9d84815a27e6ee574118aab: Deleting consensus metadata
I20260812 14:17:50.608971 9821 catalog_manager.cc:4841] TS 21bcc1fbd9d84815a27e6ee574118aab (127.8.85.3:34039): tablet 6d836ee9a2bf439aae0844312fb12323 (table test-table [id=2d655b0137d24d41bb132b852942de5d]) successfully deleted
I20260812 14:17:50.609112 9821 catalog_manager.cc:4841] TS 93ea6c9a214746258ef6005d8da97e86 (127.8.85.2:38845): tablet 6d836ee9a2bf439aae0844312fb12323 (table test-table [id=2d655b0137d24d41bb132b852942de5d]) successfully deleted
I20260812 14:17:50.609344 10490 ts_tablet_manager.cc:1905] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:50.610606 10490 ts_tablet_manager.cc:1918] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:17:50.610668 10490 log.cc:1196] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786544263511571-8532-0/minicluster-data/ts-0/wal/wals/6d836ee9a2bf439aae0844312fb12323
I20260812 14:17:50.610893 10490 ts_tablet_manager.cc:1939] T 6d836ee9a2bf439aae0844312fb12323 P a9a90b5383524e099750ac0423fb22ff: Deleting consensus metadata
I20260812 14:17:50.611186 9821 catalog_manager.cc:4841] TS a9a90b5383524e099750ac0423fb22ff (127.8.85.1:46649): tablet 6d836ee9a2bf439aae0844312fb12323 (table test-table [id=2d655b0137d24d41bb132b852942de5d]) successfully deleted
Aug 12 14:17:50 dist-test-slave-5k54 krb5kdc[9785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544269, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.85.1@KRBTEST.COM
I20260812 14:17:50.616737 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:50.608199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47312 (local address 127.8.85.1:46649)
0812 14:17:50.608241 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:50.608243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:50.608621 (+ 378us) server_negotiation.cc:407] Connection header received
0812 14:17:50.608724 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:50.608725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:50.608737 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:50.608768 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:50.609548 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.610282 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.611347 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.611620 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.611690 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:50.613935 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:50.613944 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:50.613944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:50.613952 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:50.615587 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:50.615949 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:50.615952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:50.615953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:50.616015 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:50.616221 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:50.616222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:50.616222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:50.616369 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:50.616527 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:50.616568 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 14:17:50.616645 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 14:17:50.617024 10065 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47312
I20260812 14:17:50.624066 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:50.618088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51206 (local address 127.8.85.62:42813)
0812 14:17:50.618138 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:50.618140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:50.618152 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:17:50.618231 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:50.618232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:50.618242 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:50.618290 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:50.618743 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.619297 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.620025 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:50.620277 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:50.620355 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:50.621525 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:50.621526 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:50.621527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:50.621534 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:50.622915 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:50.623250 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:50.623251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:50.623251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:50.623306 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:50.623535 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:50.623536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:50.623536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:50.623647 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:50.623776 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:50.623888 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 14:17:50.623969 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 14:17:50.624346 9835 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51206
I20260812 14:17:50.624842 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 9918
I20260812 14:17:50.634377 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 10085
I20260812 14:17:50.641019 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 10250
I20260812 14:17:50.647406 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 9800
2026-08-12T14:17:50Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2869 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:17:50 dist-test-slave-5k54 krb5kdc[10497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:17:50 dist-test-slave-5k54 krb5kdc[10497](info): set up 2 sockets
Aug 12 14:17:50 dist-test-slave-5k54 krb5kdc[10497](info): commencing operation
krb5kdc: starting...
W20260812 14:17:51.760962 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:17:51 dist-test-slave-5k54 krb5kdc[10497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:17:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:17:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:17:51.884053 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:38265
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:35501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:38265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:51.968328 10511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:51.968523 10511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:51.968544 10511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:51.969610 10511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:17:51.969642 10511 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 14:17:51.969655 10511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:51.969668 10511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:51.969681 10511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:17:51.969692 10511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:17:51.971071 10511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:35501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:38265
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:38265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:51.971257 10511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:51.971436 10511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:51.976150 10520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:51.976166 10516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:51.976318 10511 server_base.cc:1034] running on GCE node
W20260812 14:17:51.976161 10518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:51.976578 10511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:51.976907 10511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:51.978058 10511 hybrid_clock.cc:648] HybridClock initialized: now 1786544271978046 us; error 30 us; skew 500 ppm
Aug 12 14:17:51 dist-test-slave-5k54 krb5kdc[10497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544271, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:51.980595 10511 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:17:51.981063 10511 webserver.cc:460] Webserver started at http://127.8.85.62:33677/ using document root <none> and password file <none>
I20260812 14:17:51.981304 10511 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:51.981371 10511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:51.981509 10511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:51.982523 10511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "76c181579c8246c5b311c484c31237b1"
format_stamp: "Formatted at 2026-08-12 14:17:51 on dist-test-slave-5k54"
server_key: "add3123a5afb2c666b6d15774bcfd1f2"
server_key_iv: "578d672de3b6d5865cfd95c584ff2f6e"
server_key_version: "encryptionkey@0"
I20260812 14:17:51.982908 10511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "76c181579c8246c5b311c484c31237b1"
format_stamp: "Formatted at 2026-08-12 14:17:51 on dist-test-slave-5k54"
server_key: "add3123a5afb2c666b6d15774bcfd1f2"
server_key_iv: "578d672de3b6d5865cfd95c584ff2f6e"
server_key_version: "encryptionkey@0"
I20260812 14:17:51.984378 10511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:17:51.985256 10529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:51.985466 10511 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:17:51.985569 10511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "76c181579c8246c5b311c484c31237b1"
format_stamp: "Formatted at 2026-08-12 14:17:51 on dist-test-slave-5k54"
server_key: "add3123a5afb2c666b6d15774bcfd1f2"
server_key_iv: "578d672de3b6d5865cfd95c584ff2f6e"
server_key_version: "encryptionkey@0"
I20260812 14:17:51.985641 10511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:51.995890 10511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:51.998122 10511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:51.998273 10511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:52.003067 10511 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:38265
I20260812 14:17:52.003119 10599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:38265 every 8 connection(s)
I20260812 14:17:52.003499 10511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:17:52.004127 10601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:52.006382 10601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1: Bootstrap starting.
I20260812 14:17:52.006981 10601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:52.007264 10601 log.cc:824] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1: Log is configured to *not* fsync() on all Append() calls
I20260812 14:17:52.007987 10601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1: No bootstrap required, opened a new log
I20260812 14:17:52.008857 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 10511
I20260812 14:17:52.008955 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:17:52.009132 8532 external_mini_cluster.cc:1428] Setting key 87f9381070d1064c41473f5d61e5fbd8
I20260812 14:17:52.010116 10601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c181579c8246c5b311c484c31237b1" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38265 } }
I20260812 14:17:52.010290 10601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:52.010335 10601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c181579c8246c5b311c484c31237b1, State: Initialized, Role: FOLLOWER
I20260812 14:17:52.010478 10601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c181579c8246c5b311c484c31237b1" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38265 } }
I20260812 14:17:52.010571 10601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:17:52.010618 10601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:17:52.010672 10601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:52.011332 10601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c181579c8246c5b311c484c31237b1" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38265 } }
I20260812 14:17:52.011464 10601 leader_election.cc:302] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76c181579c8246c5b311c484c31237b1; no voters:
I20260812 14:17:52.011641 10601 leader_election.cc:288] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:17:52.011683 10606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:17:52.011831 10606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [term 1 LEADER]: Becoming Leader. State: Replica: 76c181579c8246c5b311c484c31237b1, State: Running, Role: LEADER
I20260812 14:17:52.011943 10601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:17:52.011977 10606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c181579c8246c5b311c484c31237b1" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38265 } }
I20260812 14:17:52.012280 10606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76c181579c8246c5b311c484c31237b1. Latest consensus state: current_term: 1 leader_uuid: "76c181579c8246c5b311c484c31237b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c181579c8246c5b311c484c31237b1" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38265 } } }
I20260812 14:17:52.012290 10609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76c181579c8246c5b311c484c31237b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c181579c8246c5b311c484c31237b1" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38265 } } }
I20260812 14:17:52.012409 10606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [sys.catalog]: This master's current role is: LEADER
I20260812 14:17:52.012476 10609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1 [sys.catalog]: This master's current role is: LEADER
I20260812 14:17:52.012686 10614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:17:52.013229 10614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:17:52.015532 10614 catalog_manager.cc:1347] Generated new cluster ID: 8b062d83b5f44a248798f198c56a7b1b
I20260812 14:17:52.015587 10614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:17:52 dist-test-slave-5k54 krb5kdc[10497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:52.020226 10605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.009636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48210 (local address 127.8.85.62:38265)
0812 14:17:52.009953 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.009961 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.009986 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:17:52.010082 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.010087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.010127 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.010239 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:52.011037 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.012328 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.013465 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.014014 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.014142 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.016587 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:52.016591 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:52.016591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:52.016603 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:52.018257 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.018638 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.018642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.018642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.019120 (+ 478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.019343 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.019344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.019344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.019640 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:52.019817 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.019897 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.020042 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":87,"threads_started":1}
I20260812 14:17:52.031185 10614 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:17:52.031919 10614 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:17:52.037449 10614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1: Generated new TSK 0
I20260812 14:17:52.037635 10614 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:17:52.068361 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38265
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:35501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:52.152004 10632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:52.152235 10632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:52.152277 10632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:52.153420 10632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:52.153499 10632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:52.153535 10632 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 14:17:52.153576 10632 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:17:52.153616 10632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:17:52.155046 10632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:35501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.85.62:38265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:52.155309 10632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:52.155579 10632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:17:52.160692 10632 server_base.cc:1034] running on GCE node
W20260812 14:17:52.160643 10638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:52.160663 10637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:52.160634 10641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:52.161067 10632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:52.161296 10632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:52.162441 10632 hybrid_clock.cc:648] HybridClock initialized: now 1786544272162425 us; error 31 us; skew 500 ppm
Aug 12 14:17:52 dist-test-slave-5k54 krb5kdc[10497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544272, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:52.165012 10632 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:17:52.165455 10632 webserver.cc:460] Webserver started at http://127.8.85.1:42465/ using document root <none> and password file <none>
I20260812 14:17:52.165654 10632 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:52.165697 10632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:52.165791 10632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:52.166762 10632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "d5b854053108417ba7f16f161ab2ba93"
format_stamp: "Formatted at 2026-08-12 14:17:52 on dist-test-slave-5k54"
server_key: "604728913cc329d781e226798e38252e"
server_key_iv: "f2d3a4f90cd7c0f7556561daae2f2918"
server_key_version: "encryptionkey@0"
I20260812 14:17:52.167068 10632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "d5b854053108417ba7f16f161ab2ba93"
format_stamp: "Formatted at 2026-08-12 14:17:52 on dist-test-slave-5k54"
server_key: "604728913cc329d781e226798e38252e"
server_key_iv: "f2d3a4f90cd7c0f7556561daae2f2918"
server_key_version: "encryptionkey@0"
I20260812 14:17:52.168375 10632 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:17:52.169008 10650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:52.169179 10632 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:52.169248 10632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "d5b854053108417ba7f16f161ab2ba93"
format_stamp: "Formatted at 2026-08-12 14:17:52 on dist-test-slave-5k54"
server_key: "604728913cc329d781e226798e38252e"
server_key_iv: "f2d3a4f90cd7c0f7556561daae2f2918"
server_key_version: "encryptionkey@0"
I20260812 14:17:52.169299 10632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:52.182297 10632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:52.184478 10632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:52.184592 10632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:52.185045 10632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:17:52.185093 10632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:52.185125 10632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:17:52.185180 10632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 14:17:52 dist-test-slave-5k54 krb5kdc[10497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544272, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:52.192235 10632 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:45443
I20260812 14:17:52.192306 10782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:45443 every 8 connection(s)
I20260812 14:17:52.192660 10632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:17:52.194067 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 10632
I20260812 14:17:52.194180 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:17:52.194368 8532 external_mini_cluster.cc:1428] Setting key 4a6d02bb16e903fdabc80c53a4120f04
I20260812 14:17:52.195747 10605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.185584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:54085 (local address 127.8.85.62:38265)
0812 14:17:52.185658 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.185661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.185823 (+ 162us) server_negotiation.cc:407] Connection header received
0812 14:17:52.186308 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.186310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.186325 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.186407 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:52.187263 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.188158 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.189700 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.190077 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.190161 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.192674 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:52.192675 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:52.192676 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:52.192684 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:52.194282 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.194782 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.194783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.194783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.194845 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.195128 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.195129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.195129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.195238 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:52.195291 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.195613 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.195684 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:17:52.196055 10660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.185539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38265 (local address 127.8.85.1:54085)
0812 14:17:52.185700 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.185740 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.186217 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.186499 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.186503 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.186839 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:52.187154 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.187161 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.188278 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.188280 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.189598 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.189601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.190169 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.190170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.190310 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.190846 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:52.190857 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:52.192573 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:52.194362 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:52.194369 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:52.194371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:52.194695 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:52.194903 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:52.194904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:52.194904 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:52.195052 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:52.195304 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:52.195311 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:52.195483 (+ 172us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.195594 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.195775 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260812 14:17:52.196599 10783 heartbeater.cc:347] Connected to a master server at 127.8.85.62:38265
I20260812 14:17:52.196703 10783 heartbeater.cc:464] Registering TS with master...
I20260812 14:17:52.197024 10783 heartbeater.cc:511] Master 127.8.85.62:38265 requested a full tablet report, sending...
I20260812 14:17:52.197714 10549 ts_manager.cc:194] Registered new tserver with Master: d5b854053108417ba7f16f161ab2ba93 (127.8.85.1:45443)
I20260812 14:17:52.198637 10549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:54085
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
I20260812 14:17:52.205061 10791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.198957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38265 (local address 127.8.85.1:41695)
0812 14:17:52.199152 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.199163 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.199224 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.199345 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.199346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.199586 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:52.199664 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.199666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.200371 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.200372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.201294 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.201297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.201624 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.201625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.201714 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.202091 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:52.202096 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:52.202529 (+ 433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:52.204075 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:52.204076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:52.204076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:52.204283 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:52.204475 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:52.204476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:52.204476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:52.204546 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:52.204789 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:52.204790 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:52.204837 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.204917 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.204992 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":76,"threads_started":1}
I20260812 14:17:52.205071 10605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.199118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:41695 (local address 127.8.85.62:38265)
0812 14:17:52.199163 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.199165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.199222 (+ 57us) server_negotiation.cc:407] Connection header received
0812 14:17:52.199270 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.199271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.199281 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.199340 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:52.199744 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.200270 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.201376 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.201554 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.201604 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.202621 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:52.202622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:52.202622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:52.202628 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:52.203997 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.204360 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.204361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.204361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.204413 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.204617 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.204618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.204618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.204715 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:52.204774 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.204953 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.205015 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:17:52.222201 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38265
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:35501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:52.304788 10794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:52.304987 10794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:52.305011 10794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:52.306177 10794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:52.306221 10794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:52.306243 10794 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 14:17:52.306255 10794 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:17:52.306299 10794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:17:52.307811 10794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:35501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.85.62:38265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:52.308059 10794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:52.308298 10794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:52.313335 10805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:52.313274 10807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:52.313484 10794 server_base.cc:1034] running on GCE node
W20260812 14:17:52.313285 10809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:52.313764 10794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:52.313995 10794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:52.315158 10794 hybrid_clock.cc:648] HybridClock initialized: now 1786544272315139 us; error 34 us; skew 500 ppm
Aug 12 14:17:52 dist-test-slave-5k54 krb5kdc[10497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544272, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:52.317687 10794 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:17:52.318125 10794 webserver.cc:460] Webserver started at http://127.8.85.2:38319/ using document root <none> and password file <none>
I20260812 14:17:52.318361 10794 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:52.318432 10794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:52.318642 10794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:52.319763 10794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "90474168f1a2437fb103fba253f8dab9"
format_stamp: "Formatted at 2026-08-12 14:17:52 on dist-test-slave-5k54"
server_key: "5ab040b5742e422be36ccd2e59c9b47c"
server_key_iv: "74010349d05c95a33a0803662e719cc6"
server_key_version: "encryptionkey@0"
I20260812 14:17:52.320128 10794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "90474168f1a2437fb103fba253f8dab9"
format_stamp: "Formatted at 2026-08-12 14:17:52 on dist-test-slave-5k54"
server_key: "5ab040b5742e422be36ccd2e59c9b47c"
server_key_iv: "74010349d05c95a33a0803662e719cc6"
server_key_version: "encryptionkey@0"
I20260812 14:17:52.321512 10794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:17:52.322265 10817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:52.322430 10794 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:17:52.322525 10794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "90474168f1a2437fb103fba253f8dab9"
format_stamp: "Formatted at 2026-08-12 14:17:52 on dist-test-slave-5k54"
server_key: "5ab040b5742e422be36ccd2e59c9b47c"
server_key_iv: "74010349d05c95a33a0803662e719cc6"
server_key_version: "encryptionkey@0"
I20260812 14:17:52.322602 10794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:52.332352 10794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:52.334537 10794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:52.334690 10794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:52.335180 10794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:17:52.335250 10794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:52.335290 10794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:17:52.335316 10794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:52.342262 10794 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:41727
I20260812 14:17:52.342329 10956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:41727 every 8 connection(s)
Aug 12 14:17:52 dist-test-slave-5k54 krb5kdc[10497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544272, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:52.342655 10794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:17:52.345351 10605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.335662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:39667 (local address 127.8.85.62:38265)
0812 14:17:52.335705 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.335708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.335931 (+ 223us) server_negotiation.cc:407] Connection header received
0812 14:17:52.336450 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.336451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.336464 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.336524 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:52.337294 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.338106 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.339846 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.340172 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.340241 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.343043 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:52.343045 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:52.343045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:52.343052 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:52.344234 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.344601 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.344602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.344602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.344654 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.344860 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.344861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.344861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.344961 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:52.345010 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.345227 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.345289 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:17:52.345569 10827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.335664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38265 (local address 127.8.85.2:39667)
0812 14:17:52.335810 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.335843 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.336330 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.336592 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.336595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.336918 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:52.337205 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.337211 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.338212 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.338213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.339732 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.339735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.340267 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.340268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.340434 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.341002 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:52.341013 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:52.342932 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:52.344300 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:52.344305 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:52.344307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:52.344535 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:52.344707 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:52.344708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:52.344708 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:52.344809 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:52.345011 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:52.345015 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:52.345117 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.345200 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.345312 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":87,"threads_started":1}
I20260812 14:17:52.346101 10957 heartbeater.cc:347] Connected to a master server at 127.8.85.62:38265
I20260812 14:17:52.346184 10957 heartbeater.cc:464] Registering TS with master...
I20260812 14:17:52.346388 10957 heartbeater.cc:511] Master 127.8.85.62:38265 requested a full tablet report, sending...
I20260812 14:17:52.346760 10549 ts_manager.cc:194] Registered new tserver with Master: 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727)
I20260812 14:17:52.347250 10549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:39667
I20260812 14:17:52.347301 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 10794
I20260812 14:17:52.347388 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:17:52.347657 8532 external_mini_cluster.cc:1428] Setting key 709a6a9f5e046801c946e70473e39e56
I20260812 14:17:52.352960 10605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.347219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:41903 (local address 127.8.85.62:38265)
0812 14:17:52.347262 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.347264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.347290 (+ 26us) server_negotiation.cc:407] Connection header received
0812 14:17:52.347446 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.347446 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.347457 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.347504 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:52.347924 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.348506 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.349149 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.349449 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.349522 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.350446 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:52.350447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:52.350448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:52.350454 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:52.351866 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.352250 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.352251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.352251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.352298 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.352508 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.352509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.352509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.352635 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:52.352685 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.352848 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.352905 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:17:52.353057 10963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.347094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38265 (local address 127.8.85.2:41903)
0812 14:17:52.347232 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.347243 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.347305 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.347544 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.347545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.347752 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:52.347824 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.347826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.348598 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.348599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.349073 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.349075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.349527 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.349527 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.349584 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.349975 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:52.349979 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:52.350323 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:52.351944 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:52.351945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:52.351946 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:52.352176 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:52.352368 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:52.352369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:52.352370 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:52.352444 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:52.352686 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:52.352687 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:52.352738 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.352834 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.352940 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:17:52.374693 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38265
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:35501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:52.458251 10966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:52.458454 10966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:52.458479 10966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:52.459635 10966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:52.459677 10966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:52.459697 10966 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 14:17:52.459709 10966 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:17:52.459726 10966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:17:52.461108 10966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:35501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.85.62:38265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:52.461284 10966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:52.461455 10966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:52.466332 10971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:52.466348 10974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:52.466348 10972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:52.466456 10966 server_base.cc:1034] running on GCE node
I20260812 14:17:52.466838 10966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:52.467095 10966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:52.468262 10966 hybrid_clock.cc:648] HybridClock initialized: now 1786544272468256 us; error 34 us; skew 500 ppm
Aug 12 14:17:52 dist-test-slave-5k54 krb5kdc[10497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544272, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:52.470736 10966 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:17:52.471235 10966 webserver.cc:460] Webserver started at http://127.8.85.3:37093/ using document root <none> and password file <none>
I20260812 14:17:52.471529 10966 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:52.471599 10966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:52.471750 10966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:52.472824 10966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "2ccf394e09f44fe3809bb2a459e31b80"
format_stamp: "Formatted at 2026-08-12 14:17:52 on dist-test-slave-5k54"
server_key: "4a359d1623ff8d8bb5ffd2df6c853a47"
server_key_iv: "860ef4beaa4e8a44e680fb1b1846c5c8"
server_key_version: "encryptionkey@0"
I20260812 14:17:52.473196 10966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "2ccf394e09f44fe3809bb2a459e31b80"
format_stamp: "Formatted at 2026-08-12 14:17:52 on dist-test-slave-5k54"
server_key: "4a359d1623ff8d8bb5ffd2df6c853a47"
server_key_iv: "860ef4beaa4e8a44e680fb1b1846c5c8"
server_key_version: "encryptionkey@0"
I20260812 14:17:52.474557 10966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:17:52.475301 10981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:52.475497 10966 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:17:52.475592 10966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "2ccf394e09f44fe3809bb2a459e31b80"
format_stamp: "Formatted at 2026-08-12 14:17:52 on dist-test-slave-5k54"
server_key: "4a359d1623ff8d8bb5ffd2df6c853a47"
server_key_iv: "860ef4beaa4e8a44e680fb1b1846c5c8"
server_key_version: "encryptionkey@0"
I20260812 14:17:52.475673 10966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:52.492987 10966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:52.494763 10966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:52.494943 10966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:52.495564 10966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:17:52.495625 10966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:52.495656 10966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:17:52.495685 10966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 14:17:52 dist-test-slave-5k54 krb5kdc[10497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544272, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:52.502938 10966 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:33931
I20260812 14:17:52.503024 11116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:33931 every 8 connection(s)
I20260812 14:17:52.503341 10966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:17:52.505487 10605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.495905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:48549 (local address 127.8.85.62:38265)
0812 14:17:52.495948 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.495950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.496168 (+ 218us) server_negotiation.cc:407] Connection header received
0812 14:17:52.496621 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.496622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.496637 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.496715 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:52.497486 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.498274 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.499983 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.500187 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.500262 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.502467 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:52.502469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:52.502470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:52.502477 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:52.504105 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.504516 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.504517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.504518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.504583 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.504844 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.504844 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.504845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.504970 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:52.505038 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.505313 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.505396 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:17:52.505674 10990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.495892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38265 (local address 127.8.85.3:48549)
0812 14:17:52.496037 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.496070 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.496527 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.496766 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.496770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.497089 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:52.497356 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.497364 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.498384 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.498385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.499893 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.499896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.500288 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.500290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.500456 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.500974 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:52.500984 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:52.502365 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:52.504202 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:52.504209 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:52.504212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:52.504440 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:52.504653 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:52.504654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:52.504654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:52.504762 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:52.505041 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:52.505048 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:52.505184 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.505286 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.505451 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260812 14:17:52.506158 11117 heartbeater.cc:347] Connected to a master server at 127.8.85.62:38265
I20260812 14:17:52.506242 11117 heartbeater.cc:464] Registering TS with master...
I20260812 14:17:52.506434 11117 heartbeater.cc:511] Master 127.8.85.62:38265 requested a full tablet report, sending...
I20260812 14:17:52.506827 10548 ts_manager.cc:194] Registered new tserver with Master: 2ccf394e09f44fe3809bb2a459e31b80 (127.8.85.3:33931)
I20260812 14:17:52.507362 10548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:48549
I20260812 14:17:52.509914 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 10966
I20260812 14:17:52.510038 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:17:52.510238 8532 external_mini_cluster.cc:1428] Setting key 601fb73c09d5a7a19fd5f8f546af106d
I20260812 14:17:52.511113 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:17:52.513245 10605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.507247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:46045 (local address 127.8.85.62:38265)
0812 14:17:52.507284 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.507286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.507601 (+ 315us) server_negotiation.cc:407] Connection header received
0812 14:17:52.507648 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.507649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.507661 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.507729 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:52.508096 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.508581 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.509238 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.509425 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.509477 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.510557 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:52.510558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:52.510558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:52.510564 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:52.512033 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.512370 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.512372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.512372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.512447 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.512739 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.512740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.512740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.512852 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:52.512936 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.513094 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.513162 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:17:52.513264 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.507185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38265 (local address 127.8.85.3:46045)
0812 14:17:52.507529 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.507541 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.507599 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.507731 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.507731 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.507952 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:52.508016 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.508018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.508668 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.508669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.509164 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.509166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.509509 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.509510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.509576 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.510044 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:52.510050 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:52.510427 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:52.512114 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:52.512115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:52.512115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:52.512287 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:52.512505 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:52.512505 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:52.512506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:52.512567 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:52.512930 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:52.512931 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:52.512973 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.513054 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.513157 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":73,"threads_started":1}
Aug 12 14:17:52 dist-test-slave-5k54 krb5kdc[10497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 14:17:52 dist-test-slave-5k54 krb5kdc[10497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:52.535794 10605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.528995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.8.85.62:38265)
0812 14:17:52.529056 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.529058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.529067 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:17:52.529098 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.529099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.529108 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.529158 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:52.529688 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.530186 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.530910 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.531091 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.531147 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.533455 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:52.533457 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:52.533457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:52.533464 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:52.534815 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.535121 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.535121 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.535122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.535173 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.535333 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.535334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.535334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.535475 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:52.535552 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.535650 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.535708 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:17:52.537304 10548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:17:52.539044 10548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:17:52.545858 11141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.541695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:45443 (local address 127.0.0.1:51256)
0812 14:17:52.541814 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.541824 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.541866 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.542796 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.542797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.542799 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:52.542844 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.542846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.543935 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.543936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.544827 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.544829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.545515 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.545516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.545605 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.545624 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.545711 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.545767 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":82,"threads_started":1}
I20260812 14:17:52.545877 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.541854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51256 (local address 127.8.85.1:45443)
0812 14:17:52.542608 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.542613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.542624 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:17:52.542666 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.542668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.542685 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.542939 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:52.543007 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.543843 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.544920 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.545433 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.545515 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.545580 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.545732 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.545768 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":96,"threads_started":1}
I20260812 14:17:52.546175 11138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.541408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:33931 (local address 127.0.0.1:40140)
0812 14:17:52.541696 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.541708 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.541777 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.542163 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.542164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.542166 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:52.542223 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.542225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.543528 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.543529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.544861 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.544864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.545883 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.545884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.545979 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.545984 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.546042 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.546086 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260812 14:17:52.546437 11140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.541499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40140 (local address 127.8.85.3:33931)
0812 14:17:52.541942 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.541947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.541959 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:17:52.542000 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.542002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.542017 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.542093 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:52.542304 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.543413 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.544970 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.545707 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.545778 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.545840 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.546288 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.546342 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":77,"threads_started":1}
I20260812 14:17:52.546865 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.541400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:41727 (local address 127.0.0.1:52014)
0812 14:17:52.541659 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.541704 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.541778 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.542099 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.542102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.542104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:52.542172 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.542174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.543783 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.543784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.544863 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.544864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.545882 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.545883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.546698 (+ 815us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.546705 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.546763 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.546802 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":543,"thread_start_us":49,"threads_started":1}
I20260812 14:17:52.546747 10707 tablet_service.cc:1467] Processing CreateTablet for tablet d958a9af677549b788bff958c97e5c77 (DEFAULT_TABLE table=test-table [id=b3f765d21545453983256a8d41ba1768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:52.547188 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.541473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52014 (local address 127.8.85.2:41727)
0812 14:17:52.541624 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.541631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.541877 (+ 246us) server_negotiation.cc:407] Connection header received
0812 14:17:52.541926 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.541928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.541944 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.542018 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:52.542252 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.543686 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.544970 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.545707 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.545778 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.545841 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.547040 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.547084 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260812 14:17:52.547196 10707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d958a9af677549b788bff958c97e5c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:52.547102 11036 tablet_service.cc:1467] Processing CreateTablet for tablet d958a9af677549b788bff958c97e5c77 (DEFAULT_TABLE table=test-table [id=b3f765d21545453983256a8d41ba1768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:52.547497 11036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d958a9af677549b788bff958c97e5c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:52.547791 10874 tablet_service.cc:1467] Processing CreateTablet for tablet d958a9af677549b788bff958c97e5c77 (DEFAULT_TABLE table=test-table [id=b3f765d21545453983256a8d41ba1768]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:52.548125 10874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d958a9af677549b788bff958c97e5c77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:52.550530 11145 tablet_bootstrap.cc:492] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93: Bootstrap starting.
I20260812 14:17:52.550875 11146 tablet_bootstrap.cc:492] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80: Bootstrap starting.
I20260812 14:17:52.551549 11145 tablet_bootstrap.cc:654] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:52.551877 11146 tablet_bootstrap.cc:654] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:52.551926 11145 log.cc:824] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93: Log is configured to *not* fsync() on all Append() calls
I20260812 14:17:52.551872 11147 tablet_bootstrap.cc:492] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9: Bootstrap starting.
I20260812 14:17:52.552237 11146 log.cc:824] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80: Log is configured to *not* fsync() on all Append() calls
I20260812 14:17:52.552659 11147 tablet_bootstrap.cc:654] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:52.552831 11145 tablet_bootstrap.cc:492] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93: No bootstrap required, opened a new log
I20260812 14:17:52.553013 11146 tablet_bootstrap.cc:492] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80: No bootstrap required, opened a new log
I20260812 14:17:52.553023 11147 log.cc:824] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9: Log is configured to *not* fsync() on all Append() calls
I20260812 14:17:52.553083 11146 ts_tablet_manager.cc:1397] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:17:52.553074 11145 ts_tablet_manager.cc:1397] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 14:17:52.553680 11147 tablet_bootstrap.cc:492] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9: No bootstrap required, opened a new log
I20260812 14:17:52.553761 11147 ts_tablet_manager.cc:1397] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:17:52.555061 11147 raft_consensus.cc:348] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } }
I20260812 14:17:52.555213 11147 raft_consensus.cc:374] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:52.555258 11147 raft_consensus.cc:729] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90474168f1a2437fb103fba253f8dab9, State: Initialized, Role: FOLLOWER
I20260812 14:17:52.555195 11145 raft_consensus.cc:348] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } }
I20260812 14:17:52.555195 11146 raft_consensus.cc:348] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } }
I20260812 14:17:52.555352 11145 raft_consensus.cc:374] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:52.555351 11146 raft_consensus.cc:374] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:52.555408 11145 raft_consensus.cc:729] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5b854053108417ba7f16f161ab2ba93, State: Initialized, Role: FOLLOWER
I20260812 14:17:52.555408 11146 raft_consensus.cc:729] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ccf394e09f44fe3809bb2a459e31b80, State: Initialized, Role: FOLLOWER
I20260812 14:17:52.555369 11147 consensus_queue.cc:260] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } }
I20260812 14:17:52.555541 11146 consensus_queue.cc:260] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } }
I20260812 14:17:52.555553 11145 consensus_queue.cc:260] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } }
I20260812 14:17:52.555742 11147 ts_tablet_manager.cc:1428] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:17:52.555827 11146 ts_tablet_manager.cc:1428] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:17:52.555827 11145 ts_tablet_manager.cc:1428] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 14:17:52.555866 10957 heartbeater.cc:503] Master 127.8.85.62:38265 was elected leader, sending a full tablet report...
I20260812 14:17:52.555941 11117 heartbeater.cc:503] Master 127.8.85.62:38265 was elected leader, sending a full tablet report...
I20260812 14:17:52.556026 10783 heartbeater.cc:503] Master 127.8.85.62:38265 was elected leader, sending a full tablet report...
W20260812 14:17:52.593217 10958 tablet.cc:2369] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:17:52.693434 10784 tablet.cc:2369] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:17:52.696300 11155 raft_consensus.cc:482] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:17:52.696506 11155 raft_consensus.cc:504] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } }
I20260812 14:17:52.696875 11155 leader_election.cc:288] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727), d5b854053108417ba7f16f161ab2ba93 (127.8.85.1:45443)
I20260812 14:17:52.700181 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.697131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:46827 (local address 127.8.85.1:45443)
0812 14:17:52.697185 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.697187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.697412 (+ 225us) server_negotiation.cc:407] Connection header received
0812 14:17:52.697478 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.697479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.697491 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.697549 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:52.697712 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.698402 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.699348 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.699815 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.699906 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.699918 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.700064 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.700107 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:17:52.700192 10990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.697071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:41727 (local address 127.8.85.3:35957)
0812 14:17:52.697116 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.697130 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.697175 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.697367 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.697368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.697370 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:52.697484 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.697487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.698456 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.698457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.699260 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.699263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.699909 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.699910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.699998 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.700001 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.700077 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.700117 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":32}
I20260812 14:17:52.700192 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.697132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:35957 (local address 127.8.85.2:41727)
0812 14:17:52.697191 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.697193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.697203 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:17:52.697253 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.697254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.697270 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.697344 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:52.697558 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.698377 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.699381 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.699816 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.699903 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.699914 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.700095 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.700132 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:17:52.700194 11156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.697071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:45443 (local address 127.8.85.3:46827)
0812 14:17:52.697323 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.697335 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.697396 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.697552 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.697553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.697554 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:52.697621 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.697623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.698502 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.698503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.699261 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.699263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.699907 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.699908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.699990 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.699993 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.700037 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.700090 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":50,"thread_start_us":114,"threads_started":1}
I20260812 14:17:52.700588 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d958a9af677549b788bff958c97e5c77" candidate_uuid: "2ccf394e09f44fe3809bb2a459e31b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90474168f1a2437fb103fba253f8dab9" is_pre_election: true
I20260812 14:17:52.700754 10898 raft_consensus.cc:2454] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ccf394e09f44fe3809bb2a459e31b80 in term 0.
I20260812 14:17:52.700791 10730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d958a9af677549b788bff958c97e5c77" candidate_uuid: "2ccf394e09f44fe3809bb2a459e31b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5b854053108417ba7f16f161ab2ba93" is_pre_election: true
I20260812 14:17:52.700930 10730 raft_consensus.cc:2454] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ccf394e09f44fe3809bb2a459e31b80 in term 0.
I20260812 14:17:52.701152 10983 leader_election.cc:302] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ccf394e09f44fe3809bb2a459e31b80, d5b854053108417ba7f16f161ab2ba93; no voters:
I20260812 14:17:52.701290 11155 raft_consensus.cc:2781] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:17:52.701344 11155 raft_consensus.cc:482] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:17:52.701370 11155 raft_consensus.cc:3037] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:52.702072 11155 raft_consensus.cc:504] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } }
I20260812 14:17:52.702212 11155 leader_election.cc:288] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [CANDIDATE]: Term 1 election: Requested vote from peers 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727), d5b854053108417ba7f16f161ab2ba93 (127.8.85.1:45443)
I20260812 14:17:52.702368 10730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d958a9af677549b788bff958c97e5c77" candidate_uuid: "2ccf394e09f44fe3809bb2a459e31b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5b854053108417ba7f16f161ab2ba93"
I20260812 14:17:52.702368 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d958a9af677549b788bff958c97e5c77" candidate_uuid: "2ccf394e09f44fe3809bb2a459e31b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90474168f1a2437fb103fba253f8dab9"
I20260812 14:17:52.702454 10898 raft_consensus.cc:3037] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:52.702453 10730 raft_consensus.cc:3037] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:52.703366 10898 raft_consensus.cc:2454] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ccf394e09f44fe3809bb2a459e31b80 in term 1.
I20260812 14:17:52.703366 10730 raft_consensus.cc:2454] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ccf394e09f44fe3809bb2a459e31b80 in term 1.
I20260812 14:17:52.704102 10984 leader_election.cc:302] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ccf394e09f44fe3809bb2a459e31b80, 90474168f1a2437fb103fba253f8dab9; no voters:
I20260812 14:17:52.704206 11155 raft_consensus.cc:2781] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:17:52.704389 11155 raft_consensus.cc:686] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 LEADER]: Becoming Leader. State: Replica: 2ccf394e09f44fe3809bb2a459e31b80, State: Running, Role: LEADER
I20260812 14:17:52.704486 11155 consensus_queue.cc:237] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } }
I20260812 14:17:52.705415 10547 catalog_manager.cc:5495] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ccf394e09f44fe3809bb2a459e31b80 (127.8.85.3). New cstate: current_term: 1 leader_uuid: "2ccf394e09f44fe3809bb2a459e31b80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } health_report { overall_health: UNKNOWN } } }
Aug 12 14:17:52 dist-test-slave-5k54 krb5kdc[10497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544271, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:52.746912 10605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.740328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48228 (local address 127.8.85.62:38265)
0812 14:17:52.740437 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.740440 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.740515 (+ 75us) server_negotiation.cc:407] Connection header received
0812 14:17:52.740575 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.740575 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.740585 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.740628 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:52.741080 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.741558 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.742208 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.742383 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.742431 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.744471 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:52.744473 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:52.744473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:52.744479 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:52.745895 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.746220 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.746220 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.746221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.746271 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:52.746441 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:52.746442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:52.746442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:52.746538 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:52.746581 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.746696 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.746800 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 14:17:52.746990 11165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.740251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38265 (local address 127.0.0.1:48228)
0812 14:17:52.740440 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.740450 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.740518 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.740644 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.740644 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.740963 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:52.741015 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.741017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.741621 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.741622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.742141 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.742143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.742442 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.742442 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.742483 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.743009 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:52.743014 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:52.744384 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:52.745960 (+ 1576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:52.745968 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:52.745970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:52.746158 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:52.746324 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:52.746325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:52.746325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:52.746380 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:52.746584 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:52.746591 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:52.746633 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.746682 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.746851 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":69,"thread_start_us":81,"threads_started":1}
I20260812 14:17:52.748924 10547 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:48228:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 14:17:52.749137 10547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:17:52.750571 10707 tablet_service.cc:1467] Processing CreateTablet for tablet d29661e0d7d740d8b11aec7a603cc6d9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=96ee623a22d8400cb938f9fb73379ffe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:17:52.750639 10874 tablet_service.cc:1467] Processing CreateTablet for tablet d29661e0d7d740d8b11aec7a603cc6d9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=96ee623a22d8400cb938f9fb73379ffe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:17:52.750633 11036 tablet_service.cc:1467] Processing CreateTablet for tablet d29661e0d7d740d8b11aec7a603cc6d9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=96ee623a22d8400cb938f9fb73379ffe]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:17:52.750788 10874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d29661e0d7d740d8b11aec7a603cc6d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:52.750788 10707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d29661e0d7d740d8b11aec7a603cc6d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:52.750803 11036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d29661e0d7d740d8b11aec7a603cc6d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:52.752179 11146 tablet_bootstrap.cc:492] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80: Bootstrap starting.
I20260812 14:17:52.752347 11145 tablet_bootstrap.cc:492] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93: Bootstrap starting.
I20260812 14:17:52.752360 11147 tablet_bootstrap.cc:492] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9: Bootstrap starting.
I20260812 14:17:52.752691 11146 tablet_bootstrap.cc:654] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:52.752729 11145 tablet_bootstrap.cc:654] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:52.753022 11147 tablet_bootstrap.cc:654] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:52.753468 11145 tablet_bootstrap.cc:492] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93: No bootstrap required, opened a new log
I20260812 14:17:52.753530 11145 ts_tablet_manager.cc:1397] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 14:17:52.753666 11146 tablet_bootstrap.cc:492] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80: No bootstrap required, opened a new log
I20260812 14:17:52.753705 11146 ts_tablet_manager.cc:1397] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:17:52.753695 11145 raft_consensus.cc:348] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.753775 11145 raft_consensus.cc:374] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:52.753798 11145 raft_consensus.cc:729] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5b854053108417ba7f16f161ab2ba93, State: Initialized, Role: FOLLOWER
I20260812 14:17:52.753849 11145 consensus_queue.cc:260] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.753841 11146 raft_consensus.cc:348] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.753942 11146 raft_consensus.cc:374] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:52.753968 11146 raft_consensus.cc:729] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ccf394e09f44fe3809bb2a459e31b80, State: Initialized, Role: FOLLOWER
I20260812 14:17:52.753979 11145 ts_tablet_manager.cc:1428] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:52.754017 11146 consensus_queue.cc:260] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.754113 11146 ts_tablet_manager.cc:1428] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:17:52.754119 11154 tablet_replica.cc:439] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } }
I20260812 14:17:52.754232 11154 tablet_replica.cc:442] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:17:52.754299 11155 tablet_replica.cc:439] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } }
I20260812 14:17:52.754377 11155 tablet_replica.cc:442] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80: This TxnStatusTablet replica's current role is: FOLLOWER
W20260812 14:17:52.754025 11118 tablet.cc:2369] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:17:52.755062 11147 tablet_bootstrap.cc:492] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9: No bootstrap required, opened a new log
I20260812 14:17:52.755108 11147 ts_tablet_manager.cc:1397] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 14:17:52.755266 11147 raft_consensus.cc:348] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.755345 11147 raft_consensus.cc:374] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:52.755366 11147 raft_consensus.cc:729] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90474168f1a2437fb103fba253f8dab9, State: Initialized, Role: FOLLOWER
I20260812 14:17:52.755450 11147 consensus_queue.cc:260] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.755535 11147 ts_tablet_manager.cc:1428] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:17:52.755653 11153 tablet_replica.cc:439] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } }
I20260812 14:17:52.755725 11153 tablet_replica.cc:442] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:17:52.759114 11154 raft_consensus.cc:482] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:17:52.759172 11154 raft_consensus.cc:504] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.759528 11154 leader_election.cc:288] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727), 2ccf394e09f44fe3809bb2a459e31b80 (127.8.85.3:33931)
I20260812 14:17:52.762422 11140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.759792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:51595 (local address 127.8.85.3:33931)
0812 14:17:52.759832 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.759834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.759902 (+ 68us) server_negotiation.cc:407] Connection header received
0812 14:17:52.759980 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.759981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.759991 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.760046 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:52.760202 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.760780 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.761708 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.762081 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.762157 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.762167 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.762321 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.762358 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:17:52.762441 11169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.759703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:33931 (local address 127.8.85.1:51595)
0812 14:17:52.759840 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.759850 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.759919 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.760051 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.760051 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.760053 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:52.760123 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.760125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.760864 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.760864 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.761625 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.761627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.762182 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.762184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.762252 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.762255 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.762316 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.762354 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":17,"thread_start_us":49,"threads_started":1}
I20260812 14:17:52.762624 11139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.759744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:43847 (local address 127.8.85.2:41727)
0812 14:17:52.759790 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.759792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.759878 (+ 86us) server_negotiation.cc:407] Connection header received
0812 14:17:52.759940 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.759941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.759951 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.760008 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:17:52.760154 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.760668 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.761716 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.762060 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.762134 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.762144 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.762534 (+ 390us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.762571 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:17:52.762629 11170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.759691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:41727 (local address 127.8.85.1:43847)
0812 14:17:52.759802 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.759813 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.759891 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.760013 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.760013 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.760016 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:17:52.760086 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.760088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.760748 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.760749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.761624 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.761627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.762398 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.762400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.762449 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.762452 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.762515 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.762552 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":26,"thread_start_us":65,"threads_started":1}
I20260812 14:17:52.762853 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d29661e0d7d740d8b11aec7a603cc6d9" candidate_uuid: "d5b854053108417ba7f16f161ab2ba93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90474168f1a2437fb103fba253f8dab9" is_pre_election: true
I20260812 14:17:52.762948 10898 raft_consensus.cc:2454] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5b854053108417ba7f16f161ab2ba93 in term 0.
I20260812 14:17:52.762931 11058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d29661e0d7d740d8b11aec7a603cc6d9" candidate_uuid: "d5b854053108417ba7f16f161ab2ba93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ccf394e09f44fe3809bb2a459e31b80" is_pre_election: true
I20260812 14:17:52.763031 11058 raft_consensus.cc:2454] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5b854053108417ba7f16f161ab2ba93 in term 0.
I20260812 14:17:52.763119 10653 leader_election.cc:302] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90474168f1a2437fb103fba253f8dab9, d5b854053108417ba7f16f161ab2ba93; no voters:
I20260812 14:17:52.763299 11154 raft_consensus.cc:2781] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:17:52.763337 11154 raft_consensus.cc:482] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:17:52.763351 11154 raft_consensus.cc:3037] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:52.764070 11154 raft_consensus.cc:504] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.764219 11154 leader_election.cc:288] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [CANDIDATE]: Term 1 election: Requested vote from peers 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727), 2ccf394e09f44fe3809bb2a459e31b80 (127.8.85.3:33931)
I20260812 14:17:52.764416 11058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d29661e0d7d740d8b11aec7a603cc6d9" candidate_uuid: "d5b854053108417ba7f16f161ab2ba93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ccf394e09f44fe3809bb2a459e31b80"
I20260812 14:17:52.764405 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d29661e0d7d740d8b11aec7a603cc6d9" candidate_uuid: "d5b854053108417ba7f16f161ab2ba93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90474168f1a2437fb103fba253f8dab9"
I20260812 14:17:52.764482 10898 raft_consensus.cc:3037] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:52.764482 11058 raft_consensus.cc:3037] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:52.765242 10898 raft_consensus.cc:2454] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5b854053108417ba7f16f161ab2ba93 in term 1.
I20260812 14:17:52.765290 11058 raft_consensus.cc:2454] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5b854053108417ba7f16f161ab2ba93 in term 1.
I20260812 14:17:52.765435 10652 leader_election.cc:302] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ccf394e09f44fe3809bb2a459e31b80, d5b854053108417ba7f16f161ab2ba93; no voters:
I20260812 14:17:52.765542 11154 raft_consensus.cc:2781] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:17:52.765690 11154 raft_consensus.cc:686] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [term 1 LEADER]: Becoming Leader. State: Replica: d5b854053108417ba7f16f161ab2ba93, State: Running, Role: LEADER
I20260812 14:17:52.765771 11154 consensus_queue.cc:237] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.766239 11171 tablet_replica.cc:439] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93: TxnStatusTablet state changed. Reason: New leader d5b854053108417ba7f16f161ab2ba93. Latest consensus state: current_term: 1 leader_uuid: "d5b854053108417ba7f16f161ab2ba93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } }
I20260812 14:17:52.766320 11171 tablet_replica.cc:442] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:17:52.766533 11174 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 14:17:52.766508 10547 catalog_manager.cc:5495] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5b854053108417ba7f16f161ab2ba93 (127.8.85.1). New cstate: current_term: 1 leader_uuid: "d5b854053108417ba7f16f161ab2ba93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } health_report { overall_health: UNKNOWN } } }
I20260812 14:17:52.766662 11174 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 14:17:52.773196 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.770434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51260 (local address 127.8.85.1:45443)
0812 14:17:52.770473 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.770475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.770502 (+ 27us) server_negotiation.cc:407] Connection header received
0812 14:17:52.770596 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.770597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.770611 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.770712 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:17:52.770855 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.771296 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.772392 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.772566 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.772644 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.772840 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:17:52.772957 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:17:52.773016 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.773100 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.773125 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:17:52.773214 11165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.770371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:45443 (local address 127.0.0.1:51260)
0812 14:17:52.770457 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.770468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.770522 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.770724 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.770725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.770727 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:17:52.770774 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.770777 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.771376 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.771377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.772313 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.772317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.772666 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.772667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.772736 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.772755 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:17:52.773024 (+ 269us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:17:52.773033 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.773088 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.773136 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 14:17:52.775204 11058 raft_consensus.cc:1261] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 FOLLOWER]: Refusing update from remote peer d5b854053108417ba7f16f161ab2ba93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:52.775278 10898 raft_consensus.cc:1261] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Refusing update from remote peer d5b854053108417ba7f16f161ab2ba93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:52.775549 11154 consensus_queue.cc:1035] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:17:52.775668 11171 consensus_queue.cc:1035] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:17:52.776757 11155 tablet_replica.cc:439] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80: TxnStatusTablet state changed. Reason: New leader d5b854053108417ba7f16f161ab2ba93. Latest consensus state: current_term: 1 leader_uuid: "d5b854053108417ba7f16f161ab2ba93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } }
I20260812 14:17:52.776849 11155 tablet_replica.cc:442] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:17:52.776854 11153 tablet_replica.cc:439] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9: TxnStatusTablet state changed. Reason: New leader d5b854053108417ba7f16f161ab2ba93. Latest consensus state: current_term: 1 leader_uuid: "d5b854053108417ba7f16f161ab2ba93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } }
I20260812 14:17:52.776937 11153 tablet_replica.cc:442] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:17:52.778240 11171 tablet_replica.cc:439] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5b854053108417ba7f16f161ab2ba93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } }
I20260812 14:17:52.778326 11171 tablet_replica.cc:442] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:17:52.778389 11171 tablet_replica.cc:439] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5b854053108417ba7f16f161ab2ba93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } }
I20260812 14:17:52.778445 11171 tablet_replica.cc:442] T d29661e0d7d740d8b11aec7a603cc6d9 P d5b854053108417ba7f16f161ab2ba93: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:17:52.778494 11178 mvcc.cc:205] Tried to move back new op lower bound from 7317685341284786176 to 7317685341249167360. Current Snapshot: MvccSnapshot[applied={T|T < 7317685341284786176}]
I20260812 14:17:52.778477 11153 tablet_replica.cc:439] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5b854053108417ba7f16f161ab2ba93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } }
I20260812 14:17:52.778556 11153 tablet_replica.cc:442] T d29661e0d7d740d8b11aec7a603cc6d9 P 90474168f1a2437fb103fba253f8dab9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:17:52.779160 11179 mvcc.cc:205] Tried to move back new op lower bound from 7317685341284786176 to 7317685341249167360. Current Snapshot: MvccSnapshot[applied={T|T < 7317685341284786176}]
I20260812 14:17:52.779223 11155 tablet_replica.cc:439] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5b854053108417ba7f16f161ab2ba93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } } }
I20260812 14:17:52.779302 11155 tablet_replica.cc:442] T d29661e0d7d740d8b11aec7a603cc6d9 P 2ccf394e09f44fe3809bb2a459e31b80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:17:52.779596 11180 mvcc.cc:205] Tried to move back new op lower bound from 7317685341284786176 to 7317685341249167360. Current Snapshot: MvccSnapshot[applied={T|T < 7317685341284786176}]
I20260812 14:17:52.783064 11140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.780655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40142 (local address 127.8.85.3:33931)
0812 14:17:52.780707 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.780709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.780719 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:17:52.780795 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.780796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.780812 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.780867 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:17:52.781019 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.781524 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.782312 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.782519 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.782588 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.782749 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:17:52.782846 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:17:52.782931 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.782970 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.782991 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:17:52.788094 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.785460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:48441 (local address 127.8.85.1:45443)
0812 14:17:52.785510 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.785512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.785522 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:17:52.785573 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.785573 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.785597 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.785647 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:17:52.785771 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.786501 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.787221 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.787544 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.787614 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.787833 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:17:52.787881 (+ 48us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:17:52.787944 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.788009 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.788030 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:17:52.788106 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.785391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:45443 (local address 127.8.85.3:48441)
0812 14:17:52.785466 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.785477 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.785520 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.785664 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.785665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.785666 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:17:52.785710 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.785711 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.786573 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.786574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.787140 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.787142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.787642 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.787643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.787710 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.787724 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:17:52.787946 (+ 222us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:17:52.787947 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.788001 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.788044 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 14:17:52.790247 10898 raft_consensus.cc:1261] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Refusing update from remote peer 2ccf394e09f44fe3809bb2a459e31b80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:52.790365 10730 raft_consensus.cc:1261] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Refusing update from remote peer 2ccf394e09f44fe3809bb2a459e31b80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:52.790503 11157 consensus_queue.cc:1035] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:17:52.790607 11155 consensus_queue.cc:1035] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:17:52.799398 11140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.796483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:54169 (local address 127.8.85.3:33931)
0812 14:17:52.796525 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:52.796528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:52.796741 (+ 213us) server_negotiation.cc:407] Connection header received
0812 14:17:52.796803 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:52.796804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:52.796816 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:52.796870 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:17:52.797029 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.797486 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.798613 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.798952 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.799014 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.799167 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:17:52.799202 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:17:52.799258 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:52.799303 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 14:17:52.799326 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:17:52.799407 11194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:52.796430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:33931 (local address 127.8.85.1:54169)
0812 14:17:52.796673 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:52.796684 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:52.796743 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:52.796873 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:52.796874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:52.796875 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:17:52.796941 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.796943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.797565 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.797566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.798531 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:52.798534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:52.799031 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:52.799032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:52.799082 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:52.799096 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:17:52.799263 (+ 167us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:17:52.799265 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:17:52.799299 (+ 34us) client_negotiation.cc:240] Negotiation successful
0812 14:17:52.799331 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":81,"threads_started":1}
I20260812 14:17:52.809146 10549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:17:52.809270 10549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:17:52.810045 10549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1: Sending DeleteTablet for 3 replicas of tablet d958a9af677549b788bff958c97e5c77
I20260812 14:17:52.810416 10707 tablet_service.cc:1514] Processing DeleteTablet for tablet d958a9af677549b788bff958c97e5c77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:52 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:51256
I20260812 14:17:52.810456 11036 tablet_service.cc:1514] Processing DeleteTablet for tablet d958a9af677549b788bff958c97e5c77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:52 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:40140
I20260812 14:17:52.810788 11198 tablet_replica.cc:330] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93: stopping tablet replica
I20260812 14:17:52.810827 10874 tablet_service.cc:1514] Processing DeleteTablet for tablet d958a9af677549b788bff958c97e5c77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:52 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:52014
I20260812 14:17:52.810945 11198 raft_consensus.cc:2229] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:17:52.811000 11198 raft_consensus.cc:2258] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:52.811209 11200 tablet_replica.cc:330] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9: stopping tablet replica
I20260812 14:17:52.811359 11200 raft_consensus.cc:2229] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:17:52.811501 11200 raft_consensus.cc:2258] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:52.811441 10549 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:17:52.811579 10549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:17:52.811839 11199 tablet_replica.cc:330] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80: stopping tablet replica
I20260812 14:17:52.812083 11199 raft_consensus.cc:2229] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:17:52.812247 11199 raft_consensus.cc:2258] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:52.812788 10874 tablet_service.cc:1467] Processing CreateTablet for tablet 5994189e7e6f45d4bef2868e9a2b9f21 (DEFAULT_TABLE table=test-table [id=43f94f49fb8642619d23b837d2d9bc4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:52.812889 10707 tablet_service.cc:1467] Processing CreateTablet for tablet 5994189e7e6f45d4bef2868e9a2b9f21 (DEFAULT_TABLE table=test-table [id=43f94f49fb8642619d23b837d2d9bc4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:52.812947 10874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5994189e7e6f45d4bef2868e9a2b9f21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:52.812973 11200 ts_tablet_manager.cc:1905] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:52.813032 10707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5994189e7e6f45d4bef2868e9a2b9f21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:52.814478 11145 tablet_bootstrap.cc:492] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93: Bootstrap starting.
I20260812 14:17:52.814522 11036 tablet_service.cc:1467] Processing CreateTablet for tablet 5994189e7e6f45d4bef2868e9a2b9f21 (DEFAULT_TABLE table=test-table [id=43f94f49fb8642619d23b837d2d9bc4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:52.814659 11036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5994189e7e6f45d4bef2868e9a2b9f21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:52.814680 11200 ts_tablet_manager.cc:1918] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:17:52.814741 11200 log.cc:1196] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/d958a9af677549b788bff958c97e5c77
I20260812 14:17:52.814872 11145 tablet_bootstrap.cc:654] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:52.815021 11200 ts_tablet_manager.cc:1939] T d958a9af677549b788bff958c97e5c77 P 90474168f1a2437fb103fba253f8dab9: Deleting consensus metadata
I20260812 14:17:52.815438 10533 catalog_manager.cc:4841] TS 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727): tablet d958a9af677549b788bff958c97e5c77 (table test-table [id=b3f765d21545453983256a8d41ba1768]) successfully deleted
I20260812 14:17:52.815510 11145 tablet_bootstrap.cc:492] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93: No bootstrap required, opened a new log
I20260812 14:17:52.815555 11145 ts_tablet_manager.cc:1397] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 14:17:52.815845 11145 raft_consensus.cc:348] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.815944 11145 raft_consensus.cc:374] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:52.815968 11145 raft_consensus.cc:729] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5b854053108417ba7f16f161ab2ba93, State: Initialized, Role: FOLLOWER
I20260812 14:17:52.816005 11145 consensus_queue.cc:260] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.816093 11145 ts_tablet_manager.cc:1428] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:52.816164 11171 raft_consensus.cc:482] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:17:52.816191 11147 tablet_bootstrap.cc:492] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9: Bootstrap starting.
I20260812 14:17:52.816197 11171 raft_consensus.cc:504] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.816311 11171 leader_election.cc:288] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727), 2ccf394e09f44fe3809bb2a459e31b80 (127.8.85.3:33931)
I20260812 14:17:52.816416 11146 tablet_bootstrap.cc:492] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80: Bootstrap starting.
I20260812 14:17:52.816639 11147 tablet_bootstrap.cc:654] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:52.816836 11146 tablet_bootstrap.cc:654] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:52.817250 11199 ts_tablet_manager.cc:1905] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:52.817350 11058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5994189e7e6f45d4bef2868e9a2b9f21" candidate_uuid: "d5b854053108417ba7f16f161ab2ba93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ccf394e09f44fe3809bb2a459e31b80" is_pre_election: true
I20260812 14:17:52.817598 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5994189e7e6f45d4bef2868e9a2b9f21" candidate_uuid: "d5b854053108417ba7f16f161ab2ba93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90474168f1a2437fb103fba253f8dab9" is_pre_election: true
W20260812 14:17:52.818115 10652 leader_election.cc:341] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2ccf394e09f44fe3809bb2a459e31b80 (127.8.85.3:33931): Illegal state: must be running to vote when last-logged opid is not known
W20260812 14:17:52.818274 10653 leader_election.cc:341] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727): Illegal state: must be running to vote when last-logged opid is not known
I20260812 14:17:52.818310 10653 leader_election.cc:302] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5b854053108417ba7f16f161ab2ba93; no voters: 2ccf394e09f44fe3809bb2a459e31b80, 90474168f1a2437fb103fba253f8dab9
I20260812 14:17:52.818411 11154 raft_consensus.cc:2726] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 14:17:52.816103 11198 ts_tablet_manager.cc:1905] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:52.819096 11146 tablet_bootstrap.cc:492] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80: No bootstrap required, opened a new log
I20260812 14:17:52.819156 11146 ts_tablet_manager.cc:1397] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 14:17:52.819291 11146 raft_consensus.cc:348] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.819360 11146 raft_consensus.cc:374] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:52.819386 11146 raft_consensus.cc:729] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ccf394e09f44fe3809bb2a459e31b80, State: Initialized, Role: FOLLOWER
I20260812 14:17:52.819478 11146 consensus_queue.cc:260] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.819599 11146 ts_tablet_manager.cc:1428] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:52.820803 11198 ts_tablet_manager.cc:1918] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:17:52.820894 11198 log.cc:1196] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/wal/wals/d958a9af677549b788bff958c97e5c77
I20260812 14:17:52.821026 11199 ts_tablet_manager.cc:1918] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:17:52.821094 11199 log.cc:1196] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/d958a9af677549b788bff958c97e5c77
I20260812 14:17:52.821419 11199 ts_tablet_manager.cc:1939] T d958a9af677549b788bff958c97e5c77 P 2ccf394e09f44fe3809bb2a459e31b80: Deleting consensus metadata
I20260812 14:17:52.821810 10532 catalog_manager.cc:4841] TS 2ccf394e09f44fe3809bb2a459e31b80 (127.8.85.3:33931): tablet d958a9af677549b788bff958c97e5c77 (table test-table [id=b3f765d21545453983256a8d41ba1768]) successfully deleted
I20260812 14:17:52.822098 11198 ts_tablet_manager.cc:1939] T d958a9af677549b788bff958c97e5c77 P d5b854053108417ba7f16f161ab2ba93: Deleting consensus metadata
I20260812 14:17:52.822237 11147 tablet_bootstrap.cc:492] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9: No bootstrap required, opened a new log
I20260812 14:17:52.822289 11147 ts_tablet_manager.cc:1397] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260812 14:17:52.822455 11147 raft_consensus.cc:348] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.822505 10532 catalog_manager.cc:4841] TS d5b854053108417ba7f16f161ab2ba93 (127.8.85.1:45443): tablet d958a9af677549b788bff958c97e5c77 (table test-table [id=b3f765d21545453983256a8d41ba1768]) successfully deleted
I20260812 14:17:52.822536 11147 raft_consensus.cc:374] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:52.822559 11147 raft_consensus.cc:729] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90474168f1a2437fb103fba253f8dab9, State: Initialized, Role: FOLLOWER
I20260812 14:17:52.822602 11147 consensus_queue.cc:260] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.822690 11147 ts_tablet_manager.cc:1428] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:52.917307 11157 raft_consensus.cc:482] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:17:52.917479 11157 raft_consensus.cc:504] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.917662 11157 leader_election.cc:288] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5b854053108417ba7f16f161ab2ba93 (127.8.85.1:45443), 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727)
I20260812 14:17:52.917873 10730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5994189e7e6f45d4bef2868e9a2b9f21" candidate_uuid: "2ccf394e09f44fe3809bb2a459e31b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5b854053108417ba7f16f161ab2ba93" is_pre_election: true
I20260812 14:17:52.917922 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5994189e7e6f45d4bef2868e9a2b9f21" candidate_uuid: "2ccf394e09f44fe3809bb2a459e31b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90474168f1a2437fb103fba253f8dab9" is_pre_election: true
I20260812 14:17:52.917989 10730 raft_consensus.cc:2454] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ccf394e09f44fe3809bb2a459e31b80 in term 0.
I20260812 14:17:52.918009 10898 raft_consensus.cc:2454] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ccf394e09f44fe3809bb2a459e31b80 in term 0.
I20260812 14:17:52.918167 10983 leader_election.cc:302] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ccf394e09f44fe3809bb2a459e31b80, d5b854053108417ba7f16f161ab2ba93; no voters:
I20260812 14:17:52.918306 11157 raft_consensus.cc:2781] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:17:52.918359 11157 raft_consensus.cc:482] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:17:52.918383 11157 raft_consensus.cc:3037] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:52.919454 11157 raft_consensus.cc:504] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.919585 11157 leader_election.cc:288] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [CANDIDATE]: Term 1 election: Requested vote from peers d5b854053108417ba7f16f161ab2ba93 (127.8.85.1:45443), 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727)
I20260812 14:17:52.919732 10730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5994189e7e6f45d4bef2868e9a2b9f21" candidate_uuid: "2ccf394e09f44fe3809bb2a459e31b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5b854053108417ba7f16f161ab2ba93"
I20260812 14:17:52.919813 10730 raft_consensus.cc:3037] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:52.919790 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5994189e7e6f45d4bef2868e9a2b9f21" candidate_uuid: "2ccf394e09f44fe3809bb2a459e31b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90474168f1a2437fb103fba253f8dab9"
I20260812 14:17:52.919853 10898 raft_consensus.cc:3037] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:52.920559 10730 raft_consensus.cc:2454] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ccf394e09f44fe3809bb2a459e31b80 in term 1.
I20260812 14:17:52.920614 10898 raft_consensus.cc:2454] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ccf394e09f44fe3809bb2a459e31b80 in term 1.
I20260812 14:17:52.920786 10984 leader_election.cc:302] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ccf394e09f44fe3809bb2a459e31b80, 90474168f1a2437fb103fba253f8dab9; no voters:
I20260812 14:17:52.920926 11157 raft_consensus.cc:2781] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:17:52.921005 11157 raft_consensus.cc:686] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 LEADER]: Becoming Leader. State: Replica: 2ccf394e09f44fe3809bb2a459e31b80, State: Running, Role: LEADER
I20260812 14:17:52.921072 11157 consensus_queue.cc:237] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:52.921728 10549 catalog_manager.cc:5495] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ccf394e09f44fe3809bb2a459e31b80 (127.8.85.3). New cstate: current_term: 1 leader_uuid: "2ccf394e09f44fe3809bb2a459e31b80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } health_report { overall_health: HEALTHY } } }
I20260812 14:17:52.938444 10730 raft_consensus.cc:1261] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Refusing update from remote peer 2ccf394e09f44fe3809bb2a459e31b80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:52.938436 10898 raft_consensus.cc:1261] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Refusing update from remote peer 2ccf394e09f44fe3809bb2a459e31b80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:52.938741 11157 consensus_queue.cc:1035] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:17:52.938863 11195 consensus_queue.cc:1035] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 14:17:52.943331 11179 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 14:17:54.461174 10548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:17:54.461301 10548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:17:54.462368 10548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1: Sending DeleteTablet for 3 replicas of tablet 5994189e7e6f45d4bef2868e9a2b9f21
I20260812 14:17:54.462641 10707 tablet_service.cc:1514] Processing DeleteTablet for tablet 5994189e7e6f45d4bef2868e9a2b9f21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:54 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:51256
I20260812 14:17:54.462647 10874 tablet_service.cc:1514] Processing DeleteTablet for tablet 5994189e7e6f45d4bef2868e9a2b9f21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:54 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:52014
I20260812 14:17:54.462913 11239 tablet_replica.cc:330] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9: stopping tablet replica
I20260812 14:17:54.462973 11239 raft_consensus.cc:2229] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:17:54.463008 11239 raft_consensus.cc:2258] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:54.463116 11237 tablet_replica.cc:330] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93: stopping tablet replica
I20260812 14:17:54.463169 11237 raft_consensus.cc:2229] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:17:54.463208 11237 raft_consensus.cc:2258] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:54.463384 11239 ts_tablet_manager.cc:1905] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:54.464975 11239 ts_tablet_manager.cc:1918] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 14:17:54.465044 11239 log.cc:1196] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/5994189e7e6f45d4bef2868e9a2b9f21
I20260812 14:17:54.465291 11239 ts_tablet_manager.cc:1939] T 5994189e7e6f45d4bef2868e9a2b9f21 P 90474168f1a2437fb103fba253f8dab9: Deleting consensus metadata
I20260812 14:17:54.465646 10533 catalog_manager.cc:4841] TS 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727): tablet 5994189e7e6f45d4bef2868e9a2b9f21 (table test-table [id=43f94f49fb8642619d23b837d2d9bc4a]) successfully deleted
I20260812 14:17:54.465889 11036 tablet_service.cc:1514] Processing DeleteTablet for tablet 5994189e7e6f45d4bef2868e9a2b9f21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:54 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:40140
I20260812 14:17:54.466123 11241 tablet_replica.cc:330] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80: stopping tablet replica
I20260812 14:17:54.466223 11241 raft_consensus.cc:2229] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:17:54.466240 11237 ts_tablet_manager.cc:1905] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:54.466321 11241 raft_consensus.cc:2258] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:54.466583 11241 ts_tablet_manager.cc:1905] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:54.468206 11237 ts_tablet_manager.cc:1918] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 14:17:54.468281 11237 log.cc:1196] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/wal/wals/5994189e7e6f45d4bef2868e9a2b9f21
I20260812 14:17:54.468397 11241 ts_tablet_manager.cc:1918] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 14:17:54.468459 11241 log.cc:1196] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/5994189e7e6f45d4bef2868e9a2b9f21
I20260812 14:17:54.468523 11237 ts_tablet_manager.cc:1939] T 5994189e7e6f45d4bef2868e9a2b9f21 P d5b854053108417ba7f16f161ab2ba93: Deleting consensus metadata
I20260812 14:17:54.468760 11241 ts_tablet_manager.cc:1939] T 5994189e7e6f45d4bef2868e9a2b9f21 P 2ccf394e09f44fe3809bb2a459e31b80: Deleting consensus metadata
I20260812 14:17:54.468997 10532 catalog_manager.cc:4841] TS d5b854053108417ba7f16f161ab2ba93 (127.8.85.1:45443): tablet 5994189e7e6f45d4bef2868e9a2b9f21 (table test-table [id=43f94f49fb8642619d23b837d2d9bc4a]) successfully deleted
I20260812 14:17:54.470139 10532 catalog_manager.cc:4841] TS 2ccf394e09f44fe3809bb2a459e31b80 (127.8.85.3:33931): tablet 5994189e7e6f45d4bef2868e9a2b9f21 (table test-table [id=43f94f49fb8642619d23b837d2d9bc4a]) successfully deleted
Aug 12 14:17:54 dist-test-slave-5k54 krb5kdc[10497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544274, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 14:17:54 dist-test-slave-5k54 krb5kdc[10497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544274, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:54.494117 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:54.480500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48720 (local address 127.8.85.62:38265)
0812 14:17:54.480650 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:54.480652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:54.480684 (+ 32us) server_negotiation.cc:407] Connection header received
0812 14:17:54.480783 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:54.480783 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:54.480796 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:54.480868 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:54.481447 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:54.482003 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:54.482767 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:54.483003 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:54.483085 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:54.491560 (+ 8475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:54.491561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:54.491562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:54.491569 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:54.492950 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:54.493290 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:54.493291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:54.493292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:54.493358 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:54.493566 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:54.493567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:54.493567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:54.493687 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:54.493799 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:54.493912 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 14:17:54.494003 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260812 14:17:54.495702 10548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:17:54.495883 10548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:17:54.497331 10707 tablet_service.cc:1467] Processing CreateTablet for tablet c8ffc7a8cc084e94b2d17119307dead6 (DEFAULT_TABLE table=test-table [id=3712949d0aca467788149071c8c98808]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:54.497331 10874 tablet_service.cc:1467] Processing CreateTablet for tablet c8ffc7a8cc084e94b2d17119307dead6 (DEFAULT_TABLE table=test-table [id=3712949d0aca467788149071c8c98808]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:54.497360 11036 tablet_service.cc:1467] Processing CreateTablet for tablet c8ffc7a8cc084e94b2d17119307dead6 (DEFAULT_TABLE table=test-table [id=3712949d0aca467788149071c8c98808]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:54.497480 10707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ffc7a8cc084e94b2d17119307dead6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:54.497491 10874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ffc7a8cc084e94b2d17119307dead6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:54.497534 11036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ffc7a8cc084e94b2d17119307dead6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:54.498579 11250 tablet_bootstrap.cc:492] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80: Bootstrap starting.
I20260812 14:17:54.498953 11251 tablet_bootstrap.cc:492] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93: Bootstrap starting.
I20260812 14:17:54.498993 11250 tablet_bootstrap.cc:654] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:54.499328 11251 tablet_bootstrap.cc:654] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:54.499708 11252 tablet_bootstrap.cc:492] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9: Bootstrap starting.
I20260812 14:17:54.499763 11250 tablet_bootstrap.cc:492] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80: No bootstrap required, opened a new log
I20260812 14:17:54.499814 11250 ts_tablet_manager.cc:1397] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:17:54.500003 11250 raft_consensus.cc:348] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.500039 11251 tablet_bootstrap.cc:492] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93: No bootstrap required, opened a new log
I20260812 14:17:54.500093 11250 raft_consensus.cc:374] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:54.500108 11252 tablet_bootstrap.cc:654] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:54.500106 11251 ts_tablet_manager.cc:1397] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:17:54.500149 11250 raft_consensus.cc:729] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ccf394e09f44fe3809bb2a459e31b80, State: Initialized, Role: FOLLOWER
I20260812 14:17:54.500226 11250 consensus_queue.cc:260] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.500221 11251 raft_consensus.cc:348] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.500296 11251 raft_consensus.cc:374] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:54.500324 11251 raft_consensus.cc:729] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5b854053108417ba7f16f161ab2ba93, State: Initialized, Role: FOLLOWER
I20260812 14:17:54.500360 11250 ts_tablet_manager.cc:1428] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:54.500406 11251 consensus_queue.cc:260] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.500562 11251 ts_tablet_manager.cc:1428] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:17:54.501039 11252 tablet_bootstrap.cc:492] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9: No bootstrap required, opened a new log
I20260812 14:17:54.501097 11252 ts_tablet_manager.cc:1397] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:17:54.501407 11252 raft_consensus.cc:348] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.501502 11252 raft_consensus.cc:374] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:54.501518 11252 raft_consensus.cc:729] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90474168f1a2437fb103fba253f8dab9, State: Initialized, Role: FOLLOWER
I20260812 14:17:54.501602 11252 consensus_queue.cc:260] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.501809 11252 ts_tablet_manager.cc:1428] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:54.556742 11226 raft_consensus.cc:482] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:17:54.556840 11226 raft_consensus.cc:504] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.557013 11226 leader_election.cc:288] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5b854053108417ba7f16f161ab2ba93 (127.8.85.1:45443), 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727)
I20260812 14:17:54.557226 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ffc7a8cc084e94b2d17119307dead6" candidate_uuid: "2ccf394e09f44fe3809bb2a459e31b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90474168f1a2437fb103fba253f8dab9" is_pre_election: true
I20260812 14:17:54.557228 10730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ffc7a8cc084e94b2d17119307dead6" candidate_uuid: "2ccf394e09f44fe3809bb2a459e31b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5b854053108417ba7f16f161ab2ba93" is_pre_election: true
I20260812 14:17:54.557335 10730 raft_consensus.cc:2454] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ccf394e09f44fe3809bb2a459e31b80 in term 0.
I20260812 14:17:54.557341 10898 raft_consensus.cc:2454] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ccf394e09f44fe3809bb2a459e31b80 in term 0.
I20260812 14:17:54.557515 10984 leader_election.cc:302] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ccf394e09f44fe3809bb2a459e31b80, 90474168f1a2437fb103fba253f8dab9; no voters:
I20260812 14:17:54.557663 11226 raft_consensus.cc:2781] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:17:54.557726 11226 raft_consensus.cc:482] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:17:54.557742 11226 raft_consensus.cc:3037] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:54.558514 11226 raft_consensus.cc:504] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.558655 11226 leader_election.cc:288] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [CANDIDATE]: Term 1 election: Requested vote from peers d5b854053108417ba7f16f161ab2ba93 (127.8.85.1:45443), 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727)
I20260812 14:17:54.558833 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ffc7a8cc084e94b2d17119307dead6" candidate_uuid: "2ccf394e09f44fe3809bb2a459e31b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90474168f1a2437fb103fba253f8dab9"
I20260812 14:17:54.558917 10898 raft_consensus.cc:3037] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:54.558856 10730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ffc7a8cc084e94b2d17119307dead6" candidate_uuid: "2ccf394e09f44fe3809bb2a459e31b80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5b854053108417ba7f16f161ab2ba93"
I20260812 14:17:54.558962 10730 raft_consensus.cc:3037] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:54.559710 10898 raft_consensus.cc:2454] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ccf394e09f44fe3809bb2a459e31b80 in term 1.
I20260812 14:17:54.559710 10730 raft_consensus.cc:2454] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ccf394e09f44fe3809bb2a459e31b80 in term 1.
I20260812 14:17:54.559896 10983 leader_election.cc:302] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ccf394e09f44fe3809bb2a459e31b80, d5b854053108417ba7f16f161ab2ba93; no voters:
I20260812 14:17:54.560042 11226 raft_consensus.cc:2781] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:17:54.560119 11226 raft_consensus.cc:686] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 LEADER]: Becoming Leader. State: Replica: 2ccf394e09f44fe3809bb2a459e31b80, State: Running, Role: LEADER
I20260812 14:17:54.560182 11226 consensus_queue.cc:237] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.560873 10549 catalog_manager.cc:5495] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ccf394e09f44fe3809bb2a459e31b80 (127.8.85.3). New cstate: current_term: 1 leader_uuid: "2ccf394e09f44fe3809bb2a459e31b80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } health_report { overall_health: HEALTHY } } }
I20260812 14:17:54.577579 11261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:54.575400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51386 (local address 127.8.85.3:33931)
0812 14:17:54.575566 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:54.575569 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:54.575583 (+ 14us) server_negotiation.cc:407] Connection header received
0812 14:17:54.575602 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:54.575603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:54.575612 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:54.575672 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:17:54.575818 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:54.576280 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:54.577013 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:54.577196 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:54.577245 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:54.577370 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:17:54.577412 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:17:54.577451 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:54.577503 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 14:17:54.577522 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":120,"threads_started":1}
I20260812 14:17:54.579566 10898 raft_consensus.cc:1261] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Refusing update from remote peer 2ccf394e09f44fe3809bb2a459e31b80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:54.579591 10730 raft_consensus.cc:1261] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Refusing update from remote peer 2ccf394e09f44fe3809bb2a459e31b80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:54.579787 11211 consensus_queue.cc:1035] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:17:54.579878 11226 consensus_queue.cc:1035] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:17:54.589748 10549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:17:54.589850 10549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:17:54.590443 10549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1: Sending DeleteTablet for 3 replicas of tablet c8ffc7a8cc084e94b2d17119307dead6
I20260812 14:17:54.590664 10707 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ffc7a8cc084e94b2d17119307dead6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:54 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:51256
I20260812 14:17:54.590680 10874 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ffc7a8cc084e94b2d17119307dead6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:54 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:52014
I20260812 14:17:54.590680 11036 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ffc7a8cc084e94b2d17119307dead6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:54 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:40140
I20260812 14:17:54.590767 11237 tablet_replica.cc:330] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93: stopping tablet replica
I20260812 14:17:54.590767 11239 tablet_replica.cc:330] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9: stopping tablet replica
I20260812 14:17:54.590830 11237 raft_consensus.cc:2229] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:17:54.590852 11239 raft_consensus.cc:2229] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:17:54.590860 11237 raft_consensus.cc:2258] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:54.590881 11239 raft_consensus.cc:2258] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:54.590945 11241 tablet_replica.cc:330] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80: stopping tablet replica
I20260812 14:17:54.591001 11241 raft_consensus.cc:2229] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:17:54.591053 11241 raft_consensus.cc:2258] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:54.591256 11239 ts_tablet_manager.cc:1905] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:54.591308 11241 ts_tablet_manager.cc:1905] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:54.591508 11237 ts_tablet_manager.cc:1905] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:54.592913 10549 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:17:54.593089 10549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:17:54.593143 11239 ts_tablet_manager.cc:1918] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:17:54.593209 11239 log.cc:1196] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/c8ffc7a8cc084e94b2d17119307dead6
I20260812 14:17:54.593212 11241 ts_tablet_manager.cc:1918] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:17:54.593274 11241 log.cc:1196] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/c8ffc7a8cc084e94b2d17119307dead6
I20260812 14:17:54.593345 11237 ts_tablet_manager.cc:1918] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:17:54.593412 11237 log.cc:1196] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-0/wal/wals/c8ffc7a8cc084e94b2d17119307dead6
I20260812 14:17:54.593442 11239 ts_tablet_manager.cc:1939] T c8ffc7a8cc084e94b2d17119307dead6 P 90474168f1a2437fb103fba253f8dab9: Deleting consensus metadata
I20260812 14:17:54.593509 11241 ts_tablet_manager.cc:1939] T c8ffc7a8cc084e94b2d17119307dead6 P 2ccf394e09f44fe3809bb2a459e31b80: Deleting consensus metadata
I20260812 14:17:54.593642 11237 ts_tablet_manager.cc:1939] T c8ffc7a8cc084e94b2d17119307dead6 P d5b854053108417ba7f16f161ab2ba93: Deleting consensus metadata
I20260812 14:17:54.593766 10533 catalog_manager.cc:4841] TS 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727): tablet c8ffc7a8cc084e94b2d17119307dead6 (table test-table [id=3712949d0aca467788149071c8c98808]) successfully deleted
I20260812 14:17:54.593818 10532 catalog_manager.cc:4841] TS 2ccf394e09f44fe3809bb2a459e31b80 (127.8.85.3:33931): tablet c8ffc7a8cc084e94b2d17119307dead6 (table test-table [id=3712949d0aca467788149071c8c98808]) successfully deleted
I20260812 14:17:54.594065 10532 catalog_manager.cc:4841] TS d5b854053108417ba7f16f161ab2ba93 (127.8.85.1:45443): tablet c8ffc7a8cc084e94b2d17119307dead6 (table test-table [id=3712949d0aca467788149071c8c98808]) successfully deleted
I20260812 14:17:54.594828 10874 tablet_service.cc:1467] Processing CreateTablet for tablet a7571fae4c0848b78f08dac00a34ae27 (DEFAULT_TABLE table=test-table [id=df5bf47c20f94d1393bbda2ce25d1cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:54.594830 10707 tablet_service.cc:1467] Processing CreateTablet for tablet a7571fae4c0848b78f08dac00a34ae27 (DEFAULT_TABLE table=test-table [id=df5bf47c20f94d1393bbda2ce25d1cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:54.594841 11036 tablet_service.cc:1467] Processing CreateTablet for tablet a7571fae4c0848b78f08dac00a34ae27 (DEFAULT_TABLE table=test-table [id=df5bf47c20f94d1393bbda2ce25d1cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:17:54.594936 10707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7571fae4c0848b78f08dac00a34ae27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:54.594941 11036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7571fae4c0848b78f08dac00a34ae27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:54.594940 10874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7571fae4c0848b78f08dac00a34ae27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:54.595995 11251 tablet_bootstrap.cc:492] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93: Bootstrap starting.
I20260812 14:17:54.596189 11252 tablet_bootstrap.cc:492] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9: Bootstrap starting.
I20260812 14:17:54.596189 11250 tablet_bootstrap.cc:492] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80: Bootstrap starting.
I20260812 14:17:54.596369 11251 tablet_bootstrap.cc:654] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:54.596519 11250 tablet_bootstrap.cc:654] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:54.596537 11252 tablet_bootstrap.cc:654] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:54.597033 11251 tablet_bootstrap.cc:492] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93: No bootstrap required, opened a new log
I20260812 14:17:54.597091 11251 ts_tablet_manager.cc:1397] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:17:54.597225 11251 raft_consensus.cc:348] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.597296 11251 raft_consensus.cc:374] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:54.597317 11251 raft_consensus.cc:729] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5b854053108417ba7f16f161ab2ba93, State: Initialized, Role: FOLLOWER
I20260812 14:17:54.597352 11252 tablet_bootstrap.cc:492] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9: No bootstrap required, opened a new log
I20260812 14:17:54.597373 11251 consensus_queue.cc:260] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.597411 11252 ts_tablet_manager.cc:1397] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:17:54.597478 11251 ts_tablet_manager.cc:1428] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:54.597563 11252 raft_consensus.cc:348] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.597658 11252 raft_consensus.cc:374] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:54.597709 11252 raft_consensus.cc:729] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90474168f1a2437fb103fba253f8dab9, State: Initialized, Role: FOLLOWER
I20260812 14:17:54.597779 11252 consensus_queue.cc:260] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.597970 11250 tablet_bootstrap.cc:492] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80: No bootstrap required, opened a new log
I20260812 14:17:54.597971 11252 ts_tablet_manager.cc:1428] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:17:54.598075 11250 ts_tablet_manager.cc:1397] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:17:54.598205 11250 raft_consensus.cc:348] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.598289 11250 raft_consensus.cc:374] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:54.598322 11250 raft_consensus.cc:729] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ccf394e09f44fe3809bb2a459e31b80, State: Initialized, Role: FOLLOWER
I20260812 14:17:54.598385 11250 consensus_queue.cc:260] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.598588 11250 ts_tablet_manager.cc:1428] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:54.736950 11220 raft_consensus.cc:482] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:17:54.737110 11220 raft_consensus.cc:504] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.737355 11220 leader_election.cc:288] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727), 2ccf394e09f44fe3809bb2a459e31b80 (127.8.85.3:33931)
I20260812 14:17:54.737627 11058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7571fae4c0848b78f08dac00a34ae27" candidate_uuid: "d5b854053108417ba7f16f161ab2ba93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ccf394e09f44fe3809bb2a459e31b80" is_pre_election: true
I20260812 14:17:54.737715 11058 raft_consensus.cc:2454] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5b854053108417ba7f16f161ab2ba93 in term 0.
I20260812 14:17:54.737648 10897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7571fae4c0848b78f08dac00a34ae27" candidate_uuid: "d5b854053108417ba7f16f161ab2ba93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90474168f1a2437fb103fba253f8dab9" is_pre_election: true
I20260812 14:17:54.737782 10897 raft_consensus.cc:2454] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5b854053108417ba7f16f161ab2ba93 in term 0.
I20260812 14:17:54.737914 10652 leader_election.cc:302] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ccf394e09f44fe3809bb2a459e31b80, d5b854053108417ba7f16f161ab2ba93; no voters:
I20260812 14:17:54.738066 11220 raft_consensus.cc:2781] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:17:54.738121 11220 raft_consensus.cc:482] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:17:54.738144 11220 raft_consensus.cc:3037] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:54.738905 11220 raft_consensus.cc:504] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.739054 11220 leader_election.cc:288] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [CANDIDATE]: Term 1 election: Requested vote from peers 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727), 2ccf394e09f44fe3809bb2a459e31b80 (127.8.85.3:33931)
I20260812 14:17:54.739231 11058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7571fae4c0848b78f08dac00a34ae27" candidate_uuid: "d5b854053108417ba7f16f161ab2ba93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ccf394e09f44fe3809bb2a459e31b80"
I20260812 14:17:54.739284 10897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7571fae4c0848b78f08dac00a34ae27" candidate_uuid: "d5b854053108417ba7f16f161ab2ba93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90474168f1a2437fb103fba253f8dab9"
I20260812 14:17:54.739336 11058 raft_consensus.cc:3037] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:54.739356 10897 raft_consensus.cc:3037] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:54.740136 11058 raft_consensus.cc:2454] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5b854053108417ba7f16f161ab2ba93 in term 1.
I20260812 14:17:54.740190 10897 raft_consensus.cc:2454] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5b854053108417ba7f16f161ab2ba93 in term 1.
I20260812 14:17:54.740350 10653 leader_election.cc:302] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90474168f1a2437fb103fba253f8dab9, d5b854053108417ba7f16f161ab2ba93; no voters:
I20260812 14:17:54.740485 11220 raft_consensus.cc:2781] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:17:54.740549 11220 raft_consensus.cc:686] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [term 1 LEADER]: Becoming Leader. State: Replica: d5b854053108417ba7f16f161ab2ba93, State: Running, Role: LEADER
I20260812 14:17:54.740595 11220 consensus_queue.cc:237] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } }
I20260812 14:17:54.741145 10548 catalog_manager.cc:5495] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5b854053108417ba7f16f161ab2ba93 (127.8.85.1). New cstate: current_term: 1 leader_uuid: "d5b854053108417ba7f16f161ab2ba93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5b854053108417ba7f16f161ab2ba93" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 } health_report { overall_health: UNKNOWN } } }
I20260812 14:17:54.782222 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:54.779774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60830 (local address 127.8.85.1:45443)
0812 14:17:54.779967 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:54.779969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:54.779976 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:17:54.779995 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:54.779996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:54.780005 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:54.780078 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:17:54.780227 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:54.780773 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:54.781532 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:54.781721 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:54.781777 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:54.781949 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:17:54.781977 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:17:54.782016 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:54.782077 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 14:17:54.782095 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":128,"threads_started":1}
I20260812 14:17:54.786821 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:54.784490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:55677 (local address 127.8.85.1:45443)
0812 14:17:54.784559 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:54.784561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:54.784650 (+ 89us) server_negotiation.cc:407] Connection header received
0812 14:17:54.784712 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:54.784713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:54.784724 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:54.784784 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:17:54.784993 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:54.785452 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:54.786157 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:54.786344 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:54.786420 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:54.786575 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:17:54.786607 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:17:54.786664 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:54.786717 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 14:17:54.786745 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":48}
I20260812 14:17:54.786827 11275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:54.784417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:45443 (local address 127.8.85.1:55677)
0812 14:17:54.784579 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:54.784589 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:54.784662 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:54.784785 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:54.784786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:54.784787 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:17:54.784900 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:54.784902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:54.785531 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:54.785532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:54.786074 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:54.786076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:54.786436 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:54.786437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:54.786485 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:54.786489 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:17:54.786666 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:17:54.786668 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:17:54.786716 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 14:17:54.786761 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":21,"thread_start_us":99,"threads_started":1}
I20260812 14:17:54.789086 11058 raft_consensus.cc:1261] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 FOLLOWER]: Refusing update from remote peer d5b854053108417ba7f16f161ab2ba93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:54.789113 10897 raft_consensus.cc:1261] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Refusing update from remote peer d5b854053108417ba7f16f161ab2ba93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:17:54.789323 11215 consensus_queue.cc:1035] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90474168f1a2437fb103fba253f8dab9" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 41727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:17:54.789554 11215 consensus_queue.cc:1035] T a7571fae4c0848b78f08dac00a34ae27 P d5b854053108417ba7f16f161ab2ba93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ccf394e09f44fe3809bb2a459e31b80" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 14:17:54.794240 11213 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 14:17:56.316237 10549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:17:56.316360 10549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:17:56.317338 10549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76c181579c8246c5b311c484c31237b1: Sending DeleteTablet for 3 replicas of tablet a7571fae4c0848b78f08dac00a34ae27
I20260812 14:17:56.317610 10706 tablet_service.cc:1514] Processing DeleteTablet for tablet a7571fae4c0848b78f08dac00a34ae27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:56 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:51256
I20260812 14:17:56.317615 10874 tablet_service.cc:1514] Processing DeleteTablet for tablet a7571fae4c0848b78f08dac00a34ae27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:56 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:52014
I20260812 14:17:56.317826 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 10632
I20260812 14:17:56.317847 11036 tablet_service.cc:1514] Processing DeleteTablet for tablet a7571fae4c0848b78f08dac00a34ae27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:17:56 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:40140
I20260812 14:17:56.321055 11305 tablet_replica.cc:330] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80: stopping tablet replica
I20260812 14:17:56.321123 11305 raft_consensus.cc:2229] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:17:56.321164 11305 raft_consensus.cc:2258] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:56.321455 11305 ts_tablet_manager.cc:1905] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:56.321781 11304 tablet_replica.cc:330] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9: stopping tablet replica
I20260812 14:17:56.321916 11304 raft_consensus.cc:2229] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:17:56.321995 11304 raft_consensus.cc:2258] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:17:56.322288 11304 ts_tablet_manager.cc:1905] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:17:56.323508 11305 ts_tablet_manager.cc:1918] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 14:17:56.323621 11305 log.cc:1196] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/a7571fae4c0848b78f08dac00a34ae27
I20260812 14:17:56.323916 11305 ts_tablet_manager.cc:1939] T a7571fae4c0848b78f08dac00a34ae27 P 2ccf394e09f44fe3809bb2a459e31b80: Deleting consensus metadata
I20260812 14:17:56.324132 11304 ts_tablet_manager.cc:1918] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 14:17:56.324196 11304 log.cc:1196] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/a7571fae4c0848b78f08dac00a34ae27
I20260812 14:17:56.324312 10532 catalog_manager.cc:4841] TS 2ccf394e09f44fe3809bb2a459e31b80 (127.8.85.3:33931): tablet a7571fae4c0848b78f08dac00a34ae27 (table test-table [id=df5bf47c20f94d1393bbda2ce25d1cf1]) successfully deleted
I20260812 14:17:56.324430 11304 ts_tablet_manager.cc:1939] T a7571fae4c0848b78f08dac00a34ae27 P 90474168f1a2437fb103fba253f8dab9: Deleting consensus metadata
I20260812 14:17:56.324965 10533 catalog_manager.cc:4841] TS 90474168f1a2437fb103fba253f8dab9 (127.8.85.2:41727): tablet a7571fae4c0848b78f08dac00a34ae27 (table test-table [id=df5bf47c20f94d1393bbda2ce25d1cf1]) successfully deleted
W20260812 14:17:56.330153 10532 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:45443 (error 108)
I20260812 14:17:56.330502 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 10794
I20260812 14:17:56.332629 11311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:56.330610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:45443 (local address 127.0.0.1:60844)
0812 14:17:56.332538 (+ 1928us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:56.332593 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:45443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1900,"thread_start_us":68,"threads_started":1}
W20260812 14:17:56.332813 10532 catalog_manager.cc:4584] TS d5b854053108417ba7f16f161ab2ba93 (127.8.85.1:45443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a7571fae4c0848b78f08dac00a34ae27: Network error: Client connection negotiation failed: client connection to 127.8.85.1:45443: connect: Connection refused (error 111)
I20260812 14:17:56.338966 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 10966
I20260812 14:17:56.346773 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 10511
2026-08-12T14:17:56Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5699 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:17:56 dist-test-slave-5k54 krb5kdc[11314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:17:56 dist-test-slave-5k54 krb5kdc[11314](info): set up 2 sockets
Aug 12 14:17:56 dist-test-slave-5k54 krb5kdc[11314](info): commencing operation
krb5kdc: starting...
W20260812 14:17:57.575598 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260812 14:17:57.718540 8532 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.143s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260812 14:17:57.977974 8532 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.259s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260812 14:17:58.192384 8532 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.214s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/test-user.keytab.
W20260812 14:17:58.800109 8532 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.608s user 0.000s sys 0.002s
Aug 12 14:17:58 dist-test-slave-5k54 krb5kdc[11314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:17:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:17:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:17:58.849742 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:35477
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:42911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:35477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:58.937378 11328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:58.937587 11328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:58.937608 11328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:58.938675 11328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:17:58.938730 11328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:58.938745 11328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:58.938757 11328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:17:58.938772 11328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:17:58.940340 11328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:42911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:35477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:35477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:58.940539 11328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:58.940708 11328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:17:58.945874 11328 server_base.cc:1034] running on GCE node
W20260812 14:17:58.945849 11333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:58.945849 11336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:58.945837 11334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:58.946477 11328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:58.946772 11328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:58.947921 11328 hybrid_clock.cc:648] HybridClock initialized: now 1786544278947911 us; error 30 us; skew 500 ppm
Aug 12 14:17:58 dist-test-slave-5k54 krb5kdc[11314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544278, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:58.950616 11328 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:17:58.951061 11328 webserver.cc:460] Webserver started at http://127.8.85.62:44567/ using document root <none> and password file <none>
I20260812 14:17:58.951306 11328 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:58.951354 11328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:58.951534 11328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:58.952581 11328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "05e93d7c43334b2bad65d90e93916d27"
format_stamp: "Formatted at 2026-08-12 14:17:58 on dist-test-slave-5k54"
server_key: "ef0d714638e089ac95e070fbf1673eee"
server_key_iv: "b1c9bbb8140b6a608336fc2aaa7e195c"
server_key_version: "encryptionkey@0"
I20260812 14:17:58.952888 11328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "05e93d7c43334b2bad65d90e93916d27"
format_stamp: "Formatted at 2026-08-12 14:17:58 on dist-test-slave-5k54"
server_key: "ef0d714638e089ac95e070fbf1673eee"
server_key_iv: "b1c9bbb8140b6a608336fc2aaa7e195c"
server_key_version: "encryptionkey@0"
I20260812 14:17:58.954178 11328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:17:58.954906 11343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:58.955078 11328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:17:58.955169 11328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "05e93d7c43334b2bad65d90e93916d27"
format_stamp: "Formatted at 2026-08-12 14:17:58 on dist-test-slave-5k54"
server_key: "ef0d714638e089ac95e070fbf1673eee"
server_key_iv: "b1c9bbb8140b6a608336fc2aaa7e195c"
server_key_version: "encryptionkey@0"
I20260812 14:17:58.955245 11328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:58.980033 11328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:58.982024 11328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:58.982182 11328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:58.986578 11328 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:35477
I20260812 14:17:58.986616 11399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:35477 every 8 connection(s)
I20260812 14:17:58.986991 11328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:17:58.987972 11400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:17:58.990286 11400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27: Bootstrap starting.
I20260812 14:17:58.990896 11400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27: Neither blocks nor log segments found. Creating new log.
I20260812 14:17:58.991204 11400 log.cc:824] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27: Log is configured to *not* fsync() on all Append() calls
I20260812 14:17:58.991969 11400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27: No bootstrap required, opened a new log
I20260812 14:17:58.993600 11400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e93d7c43334b2bad65d90e93916d27" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35477 } }
I20260812 14:17:58.993738 11400 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:17:58.993760 11400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05e93d7c43334b2bad65d90e93916d27, State: Initialized, Role: FOLLOWER
I20260812 14:17:58.993928 11400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e93d7c43334b2bad65d90e93916d27" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35477 } }
I20260812 14:17:58.993993 11400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:17:58.994015 11400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:17:58.994042 11400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:17:58.994581 11400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e93d7c43334b2bad65d90e93916d27" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35477 } }
I20260812 14:17:58.994670 11400 leader_election.cc:302] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05e93d7c43334b2bad65d90e93916d27; no voters:
I20260812 14:17:58.994869 11400 leader_election.cc:288] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:17:58.994932 11404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:17:58.995162 11400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:17:58.995335 11404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [term 1 LEADER]: Becoming Leader. State: Replica: 05e93d7c43334b2bad65d90e93916d27, State: Running, Role: LEADER
I20260812 14:17:58.995469 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 11328
I20260812 14:17:58.995507 11404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e93d7c43334b2bad65d90e93916d27" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35477 } }
I20260812 14:17:58.995563 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:17:58.995764 8532 external_mini_cluster.cc:1428] Setting key c5275b6c12caa386bfca5ad1db4d14c4
I20260812 14:17:58.995879 11405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05e93d7c43334b2bad65d90e93916d27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e93d7c43334b2bad65d90e93916d27" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35477 } } }
I20260812 14:17:58.995970 11405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [sys.catalog]: This master's current role is: LEADER
I20260812 14:17:58.996137 11407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05e93d7c43334b2bad65d90e93916d27. Latest consensus state: current_term: 1 leader_uuid: "05e93d7c43334b2bad65d90e93916d27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e93d7c43334b2bad65d90e93916d27" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35477 } } }
I20260812 14:17:58.996202 11407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27 [sys.catalog]: This master's current role is: LEADER
W20260812 14:17:58.997218 11427 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:17:58.997292 11427 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:17:58.997349 11414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:17:58.997536 11414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:17:58.998991 11414 catalog_manager.cc:1347] Generated new cluster ID: bb82328be80c4588b602288927fc43e3
I20260812 14:17:58.999050 11414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:17:58 dist-test-slave-5k54 krb5kdc[11314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:59.007823 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:58.996442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38890 (local address 127.8.85.62:35477)
0812 14:17:58.996680 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:58.996690 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:58.996724 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:17:58.996952 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:58.996960 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:58.997032 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:58.997184 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:58.997957 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:58.999351 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:59.000436 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:59.001026 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:59.001121 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:59.004012 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:59.004017 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:59.004018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:59.004031 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:59.006120 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:59.006538 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:59.006541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:59.006542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:59.006655 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:59.006877 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:59.006878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:59.006878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:59.007166 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:59.007304 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:59.007475 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 14:17:59.007609 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
I20260812 14:17:59.029749 11414 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:17:59.030488 11414 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:17:59.039538 11414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05e93d7c43334b2bad65d90e93916d27: Generated new TSK 0
I20260812 14:17:59.039767 11414 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:17:59.074849 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:35477
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:42911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:59.158704 11432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:59.158924 11432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:59.158962 11432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:59.160123 11432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:59.160192 11432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:59.160236 11432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:17:59.161640 11432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:42911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:35477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:59.161916 11432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:59.162142 11432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:59.167068 11438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:59.167075 11437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:59.167076 11440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:59.167322 11432 server_base.cc:1034] running on GCE node
I20260812 14:17:59.167572 11432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:59.167807 11432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:59.168973 11432 hybrid_clock.cc:648] HybridClock initialized: now 1786544279168964 us; error 44 us; skew 500 ppm
Aug 12 14:17:59 dist-test-slave-5k54 krb5kdc[11314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:59.171517 11432 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:17:59.171952 11432 webserver.cc:460] Webserver started at http://127.8.85.1:36199/ using document root <none> and password file <none>
I20260812 14:17:59.172132 11432 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:59.172173 11432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:59.172259 11432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:59.173218 11432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "27258271c5c74672944c2dd29c3ba046"
format_stamp: "Formatted at 2026-08-12 14:17:59 on dist-test-slave-5k54"
server_key: "dc14dea349b0fa0fc0e573600214b055"
server_key_iv: "0e7dcbd24babfaf3138e1ae7cd5c79ed"
server_key_version: "encryptionkey@0"
I20260812 14:17:59.173532 11432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "27258271c5c74672944c2dd29c3ba046"
format_stamp: "Formatted at 2026-08-12 14:17:59 on dist-test-slave-5k54"
server_key: "dc14dea349b0fa0fc0e573600214b055"
server_key_iv: "0e7dcbd24babfaf3138e1ae7cd5c79ed"
server_key_version: "encryptionkey@0"
I20260812 14:17:59.174849 11432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:17:59.175542 11451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:59.175701 11432 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:59.175779 11432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "27258271c5c74672944c2dd29c3ba046"
format_stamp: "Formatted at 2026-08-12 14:17:59 on dist-test-slave-5k54"
server_key: "dc14dea349b0fa0fc0e573600214b055"
server_key_iv: "0e7dcbd24babfaf3138e1ae7cd5c79ed"
server_key_version: "encryptionkey@0"
I20260812 14:17:59.175832 11432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:59.189174 11432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:59.190881 11432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:59.190994 11432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:59.191169 11432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:17:59.191411 11432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:17:59.191511 11432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:59.191535 11432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:17:59.191552 11432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:59.197515 11432 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:43029
I20260812 14:17:59.197574 11580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:43029 every 8 connection(s)
I20260812 14:17:59.197887 11432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:17:59.200862 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 11432
I20260812 14:17:59.200965 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:17:59.201175 8532 external_mini_cluster.cc:1428] Setting key f63ef489639ad025eacf594a283e9a7f
Aug 12 14:17:59 dist-test-slave-5k54 krb5kdc[11314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:59.208287 11583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:59.198594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:35477 (local address 127.8.85.1:40679)
0812 14:17:59.198746 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:59.198782 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:59.199284 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:59.199569 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:59.199573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:59.200134 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:59.200320 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:59.200324 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:59.201090 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:59.201091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:59.202182 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:59.202185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:59.202557 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:59.202558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:59.202705 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:59.203200 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:59.203211 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:59.205048 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:59.206899 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:59.206905 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:59.206907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:59.207120 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:59.207337 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:59.207338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:59.207338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:59.207494 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:59.207793 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:59.207799 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:59.207933 (+ 134us) client_negotiation.cc:769] Sending connection context
0812 14:17:59.208013 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 14:17:59.208131 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260812 14:17:59.208299 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:59.198598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:40679 (local address 127.8.85.62:35477)
0812 14:17:59.198662 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:59.198664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:59.198867 (+ 203us) server_negotiation.cc:407] Connection header received
0812 14:17:59.199396 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:59.199398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:59.199415 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:59.199490 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:59.200426 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:59.200998 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:59.202271 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:59.202468 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:59.202522 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:59.205148 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:59.205150 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:59.205150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:59.205159 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:59.206814 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:59.207198 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:59.207199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:59.207200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:59.207265 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:59.207561 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:59.207562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:59.207562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:59.207717 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:59.207792 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:59.208067 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 14:17:59.208164 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:17:59.208844 11581 heartbeater.cc:347] Connected to a master server at 127.8.85.62:35477
I20260812 14:17:59.208936 11581 heartbeater.cc:464] Registering TS with master...
I20260812 14:17:59.209115 11581 heartbeater.cc:511] Master 127.8.85.62:35477 requested a full tablet report, sending...
I20260812 14:17:59.209632 11362 ts_manager.cc:194] Registered new tserver with Master: 27258271c5c74672944c2dd29c3ba046 (127.8.85.1:43029)
I20260812 14:17:59.210229 11362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:40679
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:17:59.229951 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:35477
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:42911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:59.312237 11587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:59.312467 11587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:59.312507 11587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:59.313663 11587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:59.313730 11587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:59.313774 11587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:17:59.315186 11587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:42911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:35477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:59.315483 11587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:59.315722 11587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:17:59.320706 11593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:59.320770 11595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:59.320776 11592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:59.320964 11587 server_base.cc:1034] running on GCE node
I20260812 14:17:59.321159 11587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:59.321405 11587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:59.322569 11587 hybrid_clock.cc:648] HybridClock initialized: now 1786544279322556 us; error 42 us; skew 500 ppm
Aug 12 14:17:59 dist-test-slave-5k54 krb5kdc[11314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:59.325237 11587 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:17:59.325647 11587 webserver.cc:460] Webserver started at http://127.8.85.2:44545/ using document root <none> and password file <none>
I20260812 14:17:59.325876 11587 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:59.325923 11587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:59.326020 11587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:59.327186 11587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "9607a25b4f124061922891470684f44a"
format_stamp: "Formatted at 2026-08-12 14:17:59 on dist-test-slave-5k54"
server_key: "8e45c530db66da8cc17b9ab6d3a2fb95"
server_key_iv: "a5d942bd9646aa985f6a3c742edf113b"
server_key_version: "encryptionkey@0"
I20260812 14:17:59.327621 11587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "9607a25b4f124061922891470684f44a"
format_stamp: "Formatted at 2026-08-12 14:17:59 on dist-test-slave-5k54"
server_key: "8e45c530db66da8cc17b9ab6d3a2fb95"
server_key_iv: "a5d942bd9646aa985f6a3c742edf113b"
server_key_version: "encryptionkey@0"
I20260812 14:17:59.329046 11587 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:17:59.329773 11604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:59.329969 11587 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:59.330068 11587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "9607a25b4f124061922891470684f44a"
format_stamp: "Formatted at 2026-08-12 14:17:59 on dist-test-slave-5k54"
server_key: "8e45c530db66da8cc17b9ab6d3a2fb95"
server_key_iv: "a5d942bd9646aa985f6a3c742edf113b"
server_key_version: "encryptionkey@0"
I20260812 14:17:59.330155 11587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:59.356302 11587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:59.358563 11587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:59.358731 11587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:59.358999 11587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:17:59.359341 11587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:17:59.359403 11587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:59.359463 11587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:17:59.359515 11587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:59.365799 11587 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:43171
I20260812 14:17:59.365851 11735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:43171 every 8 connection(s)
I20260812 14:17:59.366220 11587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:17:59 dist-test-slave-5k54 krb5kdc[11314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:59.375362 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 11587
I20260812 14:17:59.375490 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:17:59.375757 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:59.366998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:41913 (local address 127.8.85.62:35477)
0812 14:17:59.367064 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:59.367066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:59.367278 (+ 212us) server_negotiation.cc:407] Connection header received
0812 14:17:59.367863 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:59.367864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:59.367877 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:59.367922 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:59.368837 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:59.369323 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:59.370421 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:59.370612 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:59.370661 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:59.373017 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:59.373019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:59.373019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:59.373025 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:59.374326 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:59.374715 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:59.374716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:59.374716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:59.374765 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:59.374993 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:59.374994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:59.374994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:59.375102 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:59.375167 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:59.375625 (+ 458us) server_negotiation.cc:299] Negotiation successful
0812 14:17:59.375691 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:17:59.375748 8532 external_mini_cluster.cc:1428] Setting key a46fef1af14cf0a6eb51b09cf988d1bf
I20260812 14:17:59.376075 11738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:59.366963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:35477 (local address 127.8.85.2:41913)
0812 14:17:59.367148 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:59.367190 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:59.367751 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:59.367980 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:59.367983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:59.368562 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:59.368757 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:59.368762 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:59.369428 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:59.369429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:59.370329 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:59.370331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:59.370687 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:59.370688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:59.370801 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:59.371398 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:59.371407 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:59.372929 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:59.374410 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:59.374416 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:59.374418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:59.374651 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:59.374830 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:59.374831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:59.374832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:59.374935 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:59.375195 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:59.375202 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:59.375403 (+ 201us) client_negotiation.cc:769] Sending connection context
0812 14:17:59.375608 (+ 205us) client_negotiation.cc:240] Negotiation successful
0812 14:17:59.375802 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":114,"threads_started":1}
I20260812 14:17:59.376623 11736 heartbeater.cc:347] Connected to a master server at 127.8.85.62:35477
I20260812 14:17:59.376746 11736 heartbeater.cc:464] Registering TS with master...
I20260812 14:17:59.376928 11736 heartbeater.cc:511] Master 127.8.85.62:35477 requested a full tablet report, sending...
I20260812 14:17:59.377370 11361 ts_manager.cc:194] Registered new tserver with Master: 9607a25b4f124061922891470684f44a (127.8.85.2:43171)
I20260812 14:17:59.377942 11361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:41913
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:17:59.404682 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:35477
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:42911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:17:59.489982 11741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:17:59.490181 11741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:17:59.490206 11741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:17:59.491295 11741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:17:59.491329 11741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:17:59.491354 11741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:17:59.492835 11741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:42911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:35477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:17:59.493037 11741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:17:59.493207 11741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:17:59.498679 11741 server_base.cc:1034] running on GCE node
W20260812 14:17:59.498662 11747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:59.498662 11748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:17:59.498652 11750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:17:59.499083 11741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:17:59.499282 11741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:17:59.500433 11741 hybrid_clock.cc:648] HybridClock initialized: now 1786544279500415 us; error 33 us; skew 500 ppm
Aug 12 14:17:59 dist-test-slave-5k54 krb5kdc[11314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:17:59.506125 11741 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:17:59.506650 11741 webserver.cc:460] Webserver started at http://127.8.85.3:38331/ using document root <none> and password file <none>
I20260812 14:17:59.506889 11741 fs_manager.cc:359] Metadata directory not provided
I20260812 14:17:59.506942 11741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:17:59.507125 11741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:17:59.508332 11741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "eba88c1a1eda4d64b84b774975f0eada"
format_stamp: "Formatted at 2026-08-12 14:17:59 on dist-test-slave-5k54"
server_key: "960c1754abcdf6f24f8853d6dfd4e0e7"
server_key_iv: "81371c2aac1299ef8704447c82568e08"
server_key_version: "encryptionkey@0"
I20260812 14:17:59.508708 11741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "eba88c1a1eda4d64b84b774975f0eada"
format_stamp: "Formatted at 2026-08-12 14:17:59 on dist-test-slave-5k54"
server_key: "960c1754abcdf6f24f8853d6dfd4e0e7"
server_key_iv: "81371c2aac1299ef8704447c82568e08"
server_key_version: "encryptionkey@0"
I20260812 14:17:59.510083 11741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:17:59.510824 11757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:59.511016 11741 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:17:59.511121 11741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "eba88c1a1eda4d64b84b774975f0eada"
format_stamp: "Formatted at 2026-08-12 14:17:59 on dist-test-slave-5k54"
server_key: "960c1754abcdf6f24f8853d6dfd4e0e7"
server_key_iv: "81371c2aac1299ef8704447c82568e08"
server_key_version: "encryptionkey@0"
I20260812 14:17:59.511196 11741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:17:59.552321 11741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:17:59.554565 11741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:17:59.554746 11741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:17:59.555030 11741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:17:59.555390 11741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:17:59.555473 11741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:59.555521 11741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:17:59.555555 11741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:17:59.562492 11741 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:45877
I20260812 14:17:59.562544 11888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:45877 every 8 connection(s)
I20260812 14:17:59.562899 11741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:17:59 dist-test-slave-5k54 krb5kdc[11314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544279, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:17:59.570637 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 11741
I20260812 14:17:59.570745 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:17:59.570948 8532 external_mini_cluster.cc:1428] Setting key bc263d7e81e7dcd865a279fcf5fecacd
I20260812 14:17:59.572216 11423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:59.563656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:38489 (local address 127.8.85.62:35477)
0812 14:17:59.563734 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:59.563737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:59.563955 (+ 218us) server_negotiation.cc:407] Connection header received
0812 14:17:59.564432 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:59.564433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:59.564445 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:59.564503 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:59.565445 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:59.565981 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:59.566940 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:17:59.567137 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:17:59.567191 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:59.569408 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:17:59.569410 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:17:59.569410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:17:59.569417 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:17:59.570652 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:59.571121 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:59.571122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:59.571123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:59.571208 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:17:59.571538 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:17:59.571539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:17:59.571539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:17:59.571683 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:17:59.571752 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:17:59.572084 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 14:17:59.572151 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:17:59.572410 11892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:17:59.563655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:35477 (local address 127.8.85.3:38489)
0812 14:17:59.563820 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:17:59.563862 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:17:59.564342 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:17:59.564568 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:17:59.564571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:17:59.565158 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:17:59.565353 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:59.565358 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:59.566067 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:59.566068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:59.566863 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:17:59.566865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:17:59.567211 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:17:59.567212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:17:59.567310 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:17:59.567878 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:17:59.567887 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:17:59.569290 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:17:59.570755 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:59.570761 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:59.570764 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:59.571056 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:59.571300 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:17:59.571301 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:17:59.571302 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:17:59.571477 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:17:59.571763 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:17:59.571769 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:17:59.571940 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 14:17:59.572048 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 14:17:59.572175 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260812 14:17:59.572919 11889 heartbeater.cc:347] Connected to a master server at 127.8.85.62:35477
I20260812 14:17:59.573022 11889 heartbeater.cc:464] Registering TS with master...
I20260812 14:17:59.573202 11889 heartbeater.cc:511] Master 127.8.85.62:35477 requested a full tablet report, sending...
I20260812 14:17:59.573621 11361 ts_manager.cc:194] Registered new tserver with Master: eba88c1a1eda4d64b84b774975f0eada (127.8.85.3:45877)
I20260812 14:17:59.574004 11361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:38489
I20260812 14:17:59.582644 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 14:17:59.595906 11423 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:17:59.594788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38902 (local address 127.8.85.62:35477)
0812 14:17:59.594852 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:17:59.594864 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 14:17:59.595020 (+ 156us) server_negotiation.cc:407] Connection header received
0812 14:17:59.595112 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:17:59.595115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:17:59.595129 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:17:59.595185 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:17:59.595773 (+ 588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38902: BlockingRecv error: recv got EOF from 127.0.0.1:38902 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:17:59.597414 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 11432
I20260812 14:17:59.605906 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 11587
I20260812 14:17:59.614974 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 11741
I20260812 14:17:59.621778 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 11328
2026-08-12T14:17:59Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3275 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:17:59 dist-test-slave-5k54 krb5kdc[11902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:17:59 dist-test-slave-5k54 krb5kdc[11902](info): set up 2 sockets
Aug 12 14:17:59 dist-test-slave-5k54 krb5kdc[11902](info): commencing operation
krb5kdc: starting...
W20260812 14:18:00.737468 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:18:00 dist-test-slave-5k54 krb5kdc[11902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:18:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:18:00.863524 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:46291
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:44717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:46291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:00.948150 11917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:00.948345 11917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:00.948369 11917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:00.949513 11917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:00.949558 11917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:00.949572 11917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:00.949585 11917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:00.949597 11917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:00.951141 11917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:44717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:46291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:46291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:00.951395 11917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:00.951647 11917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:00.956431 11925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:00.956451 11922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:00.956457 11923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:00.956758 11917 server_base.cc:1034] running on GCE node
I20260812 14:18:00.957041 11917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:00.957340 11917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:00.958530 11917 hybrid_clock.cc:648] HybridClock initialized: now 1786544280958502 us; error 53 us; skew 500 ppm
Aug 12 14:18:00 dist-test-slave-5k54 krb5kdc[11902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544280, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:00.961102 11917 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:18:00.961546 11917 webserver.cc:460] Webserver started at http://127.8.85.62:37043/ using document root <none> and password file <none>
I20260812 14:18:00.961790 11917 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:00.961858 11917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:00.962004 11917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:00.963078 11917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "5b1f8a8eba9b40db9607508eaeca2d5e"
format_stamp: "Formatted at 2026-08-12 14:18:00 on dist-test-slave-5k54"
server_key: "e4345a36ad2be59dbd10df7dff0f51da"
server_key_iv: "5ab9981a7554486dccb11b2203b42582"
server_key_version: "encryptionkey@0"
I20260812 14:18:00.963514 11917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "5b1f8a8eba9b40db9607508eaeca2d5e"
format_stamp: "Formatted at 2026-08-12 14:18:00 on dist-test-slave-5k54"
server_key: "e4345a36ad2be59dbd10df7dff0f51da"
server_key_iv: "5ab9981a7554486dccb11b2203b42582"
server_key_version: "encryptionkey@0"
I20260812 14:18:00.964890 11917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:18:00.965628 11932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:00.965811 11917 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:00.965901 11917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "5b1f8a8eba9b40db9607508eaeca2d5e"
format_stamp: "Formatted at 2026-08-12 14:18:00 on dist-test-slave-5k54"
server_key: "e4345a36ad2be59dbd10df7dff0f51da"
server_key_iv: "5ab9981a7554486dccb11b2203b42582"
server_key_version: "encryptionkey@0"
I20260812 14:18:00.965986 11917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:00.973793 11917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:00.975502 11917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:00.975647 11917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:00.979557 11917 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:46291
I20260812 14:18:00.979606 11994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:46291 every 8 connection(s)
I20260812 14:18:00.979951 11917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:00.980794 11996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:00.983008 11996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e: Bootstrap starting.
I20260812 14:18:00.983623 11996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:00.983865 11996 log.cc:824] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:00.984508 11996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e: No bootstrap required, opened a new log
I20260812 14:18:00.986027 11996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b1f8a8eba9b40db9607508eaeca2d5e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46291 } }
I20260812 14:18:00.986162 11996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:00.986186 11996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b1f8a8eba9b40db9607508eaeca2d5e, State: Initialized, Role: FOLLOWER
I20260812 14:18:00.986320 11996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b1f8a8eba9b40db9607508eaeca2d5e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46291 } }
I20260812 14:18:00.986402 11996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:00.986431 11996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:00.986472 11996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:00.987200 11996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b1f8a8eba9b40db9607508eaeca2d5e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46291 } }
I20260812 14:18:00.987286 11996 leader_election.cc:302] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b1f8a8eba9b40db9607508eaeca2d5e; no voters:
I20260812 14:18:00.987406 11996 leader_election.cc:288] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:00.987506 11999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:00.987677 11999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [term 1 LEADER]: Becoming Leader. State: Replica: 5b1f8a8eba9b40db9607508eaeca2d5e, State: Running, Role: LEADER
I20260812 14:18:00.987751 11996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:00.987864 11999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b1f8a8eba9b40db9607508eaeca2d5e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46291 } }
I20260812 14:18:00.988247 12000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b1f8a8eba9b40db9607508eaeca2d5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b1f8a8eba9b40db9607508eaeca2d5e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46291 } } }
I20260812 14:18:00.988472 12000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:00.988409 12001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b1f8a8eba9b40db9607508eaeca2d5e. Latest consensus state: current_term: 1 leader_uuid: "5b1f8a8eba9b40db9607508eaeca2d5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b1f8a8eba9b40db9607508eaeca2d5e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46291 } } }
I20260812 14:18:00.988708 12001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:00.988806 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 11917
I20260812 14:18:00.988910 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:00.989012 12010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:00.989131 8532 external_mini_cluster.cc:1428] Setting key ce1e701c8701cfb7973af557d5257bf0
I20260812 14:18:00.989540 12010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:00.991357 12010 catalog_manager.cc:1347] Generated new cluster ID: 289bccb234af42abbf73d78279d3af2c
I20260812 14:18:00.991459 12010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:18:00 dist-test-slave-5k54 krb5kdc[11902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:01.001109 12022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:00.989610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53928 (local address 127.8.85.62:46291)
0812 14:18:00.989803 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:00.989813 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:00.989851 (+ 38us) server_negotiation.cc:407] Connection header received
0812 14:18:00.989979 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:00.989986 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:00.990044 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:00.990188 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:00.990987 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:00.992672 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:00.993788 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:00.994415 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:00.994510 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:00.997379 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:00.997384 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:00.997385 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:00.997397 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:00.999415 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:00.999833 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:00.999837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:00.999837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:00.999951 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:01.000169 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:01.000170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:01.000170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:01.000493 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:01.000667 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:01.000742 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 14:18:01.000896 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260812 14:18:01.002099 12010 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:01.002161 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:18:01.002564 12010 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:01.008239 12010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b1f8a8eba9b40db9607508eaeca2d5e: Generated new TSK 0
I20260812 14:18:01.008406 12010 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:18:01.024399 12022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:01.019934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53938 (local address 127.8.85.62:46291)
0812 14:18:01.020010 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:01.020012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:01.020131 (+ 119us) server_negotiation.cc:407] Connection header received
0812 14:18:01.020215 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:01.020216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:01.020225 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:01.020267 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:01.020732 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:01.021272 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:01.022383 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:01.022601 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:01.022661 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:01.023713 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:01.023716 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:01.023716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:01.023964 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:01.023986 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:01.023995 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:01.024056 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:01.024142 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 14:18:01.024194 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 14:18:01.027328 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 11917
2026-08-12T14:18:01Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1403 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:18:01 dist-test-slave-5k54 krb5kdc[12035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:18:01 dist-test-slave-5k54 krb5kdc[12035](info): set up 2 sockets
Aug 12 14:18:01 dist-test-slave-5k54 krb5kdc[12035](info): commencing operation
krb5kdc: starting...
W20260812 14:18:02.142757 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:18:02 dist-test-slave-5k54 krb5kdc[12035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:18:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:18:02.286857 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:36181
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:36751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:36181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:02.387753 12051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:02.387964 12051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:02.387985 12051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:02.389056 12051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:02.389089 12051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:02.389104 12051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:02.389117 12051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:02.389128 12051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:02.390537 12051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:36181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:36181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:02.390729 12051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:02.390894 12051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:02.395660 12060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:02.395660 12056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:02.395730 12057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:02.396003 12051 server_base.cc:1034] running on GCE node
I20260812 14:18:02.396271 12051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:02.396555 12051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:02.397727 12051 hybrid_clock.cc:648] HybridClock initialized: now 1786544282397710 us; error 37 us; skew 500 ppm
Aug 12 14:18:02 dist-test-slave-5k54 krb5kdc[12035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544282, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:02.400343 12051 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:18:02.400774 12051 webserver.cc:460] Webserver started at http://127.8.85.62:36109/ using document root <none> and password file <none>
I20260812 14:18:02.401014 12051 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:02.401060 12051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:02.401224 12051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:02.402256 12051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "4b0728d96b5a4d51a7c0b83276c8d72c"
format_stamp: "Formatted at 2026-08-12 14:18:02 on dist-test-slave-5k54"
server_key: "04bc550955034d6ad4fab4014f9c58bd"
server_key_iv: "25ec109f0051ecb40cefd33927306fcb"
server_key_version: "encryptionkey@0"
I20260812 14:18:02.402657 12051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "4b0728d96b5a4d51a7c0b83276c8d72c"
format_stamp: "Formatted at 2026-08-12 14:18:02 on dist-test-slave-5k54"
server_key: "04bc550955034d6ad4fab4014f9c58bd"
server_key_iv: "25ec109f0051ecb40cefd33927306fcb"
server_key_version: "encryptionkey@0"
I20260812 14:18:02.404165 12051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:18:02.404947 12068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:02.405162 12051 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:02.405293 12051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "4b0728d96b5a4d51a7c0b83276c8d72c"
format_stamp: "Formatted at 2026-08-12 14:18:02 on dist-test-slave-5k54"
server_key: "04bc550955034d6ad4fab4014f9c58bd"
server_key_iv: "25ec109f0051ecb40cefd33927306fcb"
server_key_version: "encryptionkey@0"
I20260812 14:18:02.405377 12051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:02.423635 12051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:02.425555 12051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:02.425727 12051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:02.429812 12051 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:36181
I20260812 14:18:02.429849 12125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:36181 every 8 connection(s)
I20260812 14:18:02.430228 12051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:02.431514 12126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:02.432924 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 12051
I20260812 14:18:02.433051 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:02.433307 8532 external_mini_cluster.cc:1428] Setting key 2e967f237f296740fed09e2b65b67297
I20260812 14:18:02.434270 12126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c: Bootstrap starting.
I20260812 14:18:02.434918 12126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:02.435259 12126 log.cc:824] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:02.436094 12126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c: No bootstrap required, opened a new log
I20260812 14:18:02.438295 12126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b0728d96b5a4d51a7c0b83276c8d72c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 36181 } }
I20260812 14:18:02.438519 12126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:02.438563 12126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b0728d96b5a4d51a7c0b83276c8d72c, State: Initialized, Role: FOLLOWER
I20260812 14:18:02.438762 12126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b0728d96b5a4d51a7c0b83276c8d72c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 36181 } }
I20260812 14:18:02.438900 12126 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:02.438939 12126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:02.438994 12126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:02.439779 12126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b0728d96b5a4d51a7c0b83276c8d72c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 36181 } }
I20260812 14:18:02.439894 12126 leader_election.cc:302] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b0728d96b5a4d51a7c0b83276c8d72c; no voters:
I20260812 14:18:02.440073 12126 leader_election.cc:288] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:02.440147 12132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:02.440342 12126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:02.440388 12132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [term 1 LEADER]: Becoming Leader. State: Replica: 4b0728d96b5a4d51a7c0b83276c8d72c, State: Running, Role: LEADER
I20260812 14:18:02.440609 12132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b0728d96b5a4d51a7c0b83276c8d72c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 36181 } }
Aug 12 14:18:02 dist-test-slave-5k54 krb5kdc[12035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:02.440977 12133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b0728d96b5a4d51a7c0b83276c8d72c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b0728d96b5a4d51a7c0b83276c8d72c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 36181 } } }
I20260812 14:18:02.441077 12133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:02.441304 12134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b0728d96b5a4d51a7c0b83276c8d72c. Latest consensus state: current_term: 1 leader_uuid: "4b0728d96b5a4d51a7c0b83276c8d72c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b0728d96b5a4d51a7c0b83276c8d72c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 36181 } } }
I20260812 14:18:02.441315 12144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:02.441406 12134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:02.441916 12144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:02.443496 12144 catalog_manager.cc:1347] Generated new cluster ID: 265f11ec2b9c4d69896400064bf94326
I20260812 14:18:02.443563 12144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:18:02.445863 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:02.433890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59720 (local address 127.8.85.62:36181)
0812 14:18:02.434061 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:02.434078 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:02.434105 (+ 27us) server_negotiation.cc:407] Connection header received
0812 14:18:02.434600 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:02.434608 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:02.434671 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:02.434796 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:02.435681 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:02.436746 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:02.437835 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:02.438280 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:02.438366 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:02.441445 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:02.441450 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:02.441451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:02.441463 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:02.444138 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:02.444540 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:02.444543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:02.444543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:02.444649 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:02.444869 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:02.444871 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:02.444871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:02.445192 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:02.445358 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:02.445498 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 14:18:02.445643 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260812 14:18:02.456022 12144 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:02.456758 12144 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:02.461076 12144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b0728d96b5a4d51a7c0b83276c8d72c: Generated new TSK 0
I20260812 14:18:02.461269 12144 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:18:02.494320 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:36181
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:36751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:02.578310 12158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:02.578508 12158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:02.578531 12158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:02.579828 12158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:02.579869 12158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:02.579919 12158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:18:02.581344 12158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:36181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:02.581599 12158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:02.581845 12158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:02.586987 12163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:02.587134 12166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:02.587008 12164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:02.587149 12158 server_base.cc:1034] running on GCE node
I20260812 14:18:02.587558 12158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:02.587795 12158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:02.588976 12158 hybrid_clock.cc:648] HybridClock initialized: now 1786544282588959 us; error 40 us; skew 500 ppm
Aug 12 14:18:02 dist-test-slave-5k54 krb5kdc[12035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544282, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:02.591668 12158 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:18:02.592085 12158 webserver.cc:460] Webserver started at http://127.8.85.1:40273/ using document root <none> and password file <none>
I20260812 14:18:02.592329 12158 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:02.592396 12158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:02.592553 12158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:02.593577 12158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "4218f8649bee4a69b21e1083d51070bd"
format_stamp: "Formatted at 2026-08-12 14:18:02 on dist-test-slave-5k54"
server_key: "26d7087541f54d716f0159f66aa359e4"
server_key_iv: "3fb80430bc91051cb58615c470a899a5"
server_key_version: "encryptionkey@0"
I20260812 14:18:02.593940 12158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "4218f8649bee4a69b21e1083d51070bd"
format_stamp: "Formatted at 2026-08-12 14:18:02 on dist-test-slave-5k54"
server_key: "26d7087541f54d716f0159f66aa359e4"
server_key_iv: "3fb80430bc91051cb58615c470a899a5"
server_key_version: "encryptionkey@0"
I20260812 14:18:02.595279 12158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:18:02.596097 12175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:02.596261 12158 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:02.596356 12158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "4218f8649bee4a69b21e1083d51070bd"
format_stamp: "Formatted at 2026-08-12 14:18:02 on dist-test-slave-5k54"
server_key: "26d7087541f54d716f0159f66aa359e4"
server_key_iv: "3fb80430bc91051cb58615c470a899a5"
server_key_version: "encryptionkey@0"
I20260812 14:18:02.596449 12158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:02.607343 12158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:02.609453 12158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:02.609606 12158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:02.609828 12158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:02.610127 12158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:02.610179 12158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:02.610220 12158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:02.610260 12158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:02.615921 12158 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:40295
I20260812 14:18:02.616024 12306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:40295 every 8 connection(s)
I20260812 14:18:02.616303 12158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:18:02.619946 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 12158
I20260812 14:18:02.620049 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:18:02.620263 8532 external_mini_cluster.cc:1428] Setting key 0cfd225f6bdf675b452b73dc408973ce
Aug 12 14:18:02 dist-test-slave-5k54 krb5kdc[12035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544282, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:02.626994 12309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:02.616959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:36181 (local address 127.8.85.1:51717)
0812 14:18:02.617135 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:02.617170 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:02.617639 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:02.617854 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:02.617857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:02.618402 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:02.618585 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:02.618590 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:02.619378 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:02.619380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:02.620711 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:02.620713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:02.621177 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:02.621178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:02.621324 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:02.621842 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:02.621853 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:02.623771 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:02.625630 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:02.625637 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:02.625639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:02.625848 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:02.626106 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:02.626107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:02.626107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:02.626206 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:02.626504 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:02.626508 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:02.626619 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 14:18:02.626719 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 14:18:02.626832 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":97,"threads_started":1}
I20260812 14:18:02.627027 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:02.616989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:51717 (local address 127.8.85.62:36181)
0812 14:18:02.617061 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:02.617064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:02.617255 (+ 191us) server_negotiation.cc:407] Connection header received
0812 14:18:02.617717 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:02.617718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:02.617728 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:02.617781 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:02.618660 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:02.619269 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:02.620787 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:02.621086 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:02.621142 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:02.623877 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:02.623879 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:02.623880 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:02.623889 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:02.625531 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:02.625957 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:02.625958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:02.625959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:02.626029 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:02.626284 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:02.626285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:02.626286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:02.626432 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:02.626509 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:02.626773 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 14:18:02.626876 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 14:18:02.627604 12307 heartbeater.cc:347] Connected to a master server at 127.8.85.62:36181
I20260812 14:18:02.627713 12307 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:02.627894 12307 heartbeater.cc:511] Master 127.8.85.62:36181 requested a full tablet report, sending...
I20260812 14:18:02.628530 12086 ts_manager.cc:194] Registered new tserver with Master: 4218f8649bee4a69b21e1083d51070bd (127.8.85.1:40295)
I20260812 14:18:02.629482 12086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:51717
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:18:02.651182 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:36181
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:36751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:02.733719 12312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:02.733979 12312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:02.734022 12312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:02.735196 12312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:02.735263 12312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:02.735316 12312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:18:02.736796 12312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:36181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:02.737053 12312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:02.737279 12312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:02.742338 12317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:02.742378 12318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:02.742338 12320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:02.742604 12312 server_base.cc:1034] running on GCE node
I20260812 14:18:02.742808 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:02.743059 12312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:02.744220 12312 hybrid_clock.cc:648] HybridClock initialized: now 1786544282744214 us; error 36 us; skew 500 ppm
Aug 12 14:18:02 dist-test-slave-5k54 krb5kdc[12035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544282, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:02.746752 12312 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:18:02.747188 12312 webserver.cc:460] Webserver started at http://127.8.85.2:40729/ using document root <none> and password file <none>
I20260812 14:18:02.747417 12312 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:02.747542 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:02.747697 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:02.748747 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "4fa736ef156740ac962a41c00b0bad1d"
format_stamp: "Formatted at 2026-08-12 14:18:02 on dist-test-slave-5k54"
server_key: "dc4d41b2cf3acafe54b0ddbb67bdc9c8"
server_key_iv: "34fdfa53d798f2b1d16382f631bb0f3e"
server_key_version: "encryptionkey@0"
I20260812 14:18:02.749112 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "4fa736ef156740ac962a41c00b0bad1d"
format_stamp: "Formatted at 2026-08-12 14:18:02 on dist-test-slave-5k54"
server_key: "dc4d41b2cf3acafe54b0ddbb67bdc9c8"
server_key_iv: "34fdfa53d798f2b1d16382f631bb0f3e"
server_key_version: "encryptionkey@0"
I20260812 14:18:02.750475 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:18:02.751204 12330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:02.751379 12312 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:02.751513 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "4fa736ef156740ac962a41c00b0bad1d"
format_stamp: "Formatted at 2026-08-12 14:18:02 on dist-test-slave-5k54"
server_key: "dc4d41b2cf3acafe54b0ddbb67bdc9c8"
server_key_iv: "34fdfa53d798f2b1d16382f631bb0f3e"
server_key_version: "encryptionkey@0"
I20260812 14:18:02.751595 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:02.792682 12312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:02.795148 12312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:02.795317 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:02.795624 12312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:02.795964 12312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:02.796023 12312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:02.796066 12312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:02.796092 12312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:02.802187 12312 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:35061
I20260812 14:18:02.802263 12457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:35061 every 8 connection(s)
I20260812 14:18:02.802619 12312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:18:02.806988 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 12312
I20260812 14:18:02.807132 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:18:02.807340 8532 external_mini_cluster.cc:1428] Setting key f6676b98e510e0d47e9af7914d97e3e2
Aug 12 14:18:02 dist-test-slave-5k54 krb5kdc[12035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544282, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:02.812698 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:02.803393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:58179 (local address 127.8.85.62:36181)
0812 14:18:02.803482 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:02.803485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:02.803692 (+ 207us) server_negotiation.cc:407] Connection header received
0812 14:18:02.804179 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:02.804180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:02.804190 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:02.804248 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:02.805201 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:02.805689 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:02.806669 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:02.807026 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:02.807184 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:02.809699 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:02.809701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:02.809701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:02.809709 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:02.811143 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:02.811652 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:02.811653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:02.811653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:02.811715 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:02.812005 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:02.812006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:02.812007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:02.812121 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:02.812197 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:02.812525 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 14:18:02.812609 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:18:02.812886 12460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:02.803353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:36181 (local address 127.8.85.2:58179)
0812 14:18:02.803550 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:02.803594 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:02.804091 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:02.804313 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:02.804316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:02.804917 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:02.805119 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:02.805125 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:02.805777 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:02.805778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:02.806591 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:02.806592 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:02.807197 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:02.807199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:02.807349 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:02.807971 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:02.807981 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:02.809599 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:02.811248 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:02.811257 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:02.811259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:02.811576 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:02.811798 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:02.811799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:02.811800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:02.811931 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:02.812189 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:02.812197 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:02.812370 (+ 173us) client_negotiation.cc:769] Sending connection context
0812 14:18:02.812487 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 14:18:02.812648 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":119,"threads_started":1}
I20260812 14:18:02.813544 12458 heartbeater.cc:347] Connected to a master server at 127.8.85.62:36181
I20260812 14:18:02.813657 12458 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:02.813869 12458 heartbeater.cc:511] Master 127.8.85.62:36181 requested a full tablet report, sending...
I20260812 14:18:02.814318 12086 ts_manager.cc:194] Registered new tserver with Master: 4fa736ef156740ac962a41c00b0bad1d (127.8.85.2:35061)
I20260812 14:18:02.814704 12086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:58179
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:18:02.834712 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:36181
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:36751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:02.921368 12463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:02.921607 12463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:02.921648 12463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:02.922817 12463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:02.922889 12463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:02.922935 12463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:18:02.924451 12463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:36181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:02.924721 12463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:02.924985 12463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:02.930080 12473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:02.930073 12470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:02.930073 12468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:02.930421 12463 server_base.cc:1034] running on GCE node
I20260812 14:18:02.930630 12463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:02.930826 12463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:02.931965 12463 hybrid_clock.cc:648] HybridClock initialized: now 1786544282931941 us; error 40 us; skew 500 ppm
Aug 12 14:18:02 dist-test-slave-5k54 krb5kdc[12035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544282, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:02.934412 12463 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:18:02.934897 12463 webserver.cc:460] Webserver started at http://127.8.85.3:43359/ using document root <none> and password file <none>
I20260812 14:18:02.935168 12463 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:02.935236 12463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:02.935405 12463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:02.936597 12463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "5957ec96398e422b94e2b958ce4c5099"
format_stamp: "Formatted at 2026-08-12 14:18:02 on dist-test-slave-5k54"
server_key: "1d59b1934eb595f4019cebe4fdda88c4"
server_key_iv: "63d858e58843dafd9d2d905d561961bb"
server_key_version: "encryptionkey@0"
I20260812 14:18:02.936985 12463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "5957ec96398e422b94e2b958ce4c5099"
format_stamp: "Formatted at 2026-08-12 14:18:02 on dist-test-slave-5k54"
server_key: "1d59b1934eb595f4019cebe4fdda88c4"
server_key_iv: "63d858e58843dafd9d2d905d561961bb"
server_key_version: "encryptionkey@0"
I20260812 14:18:02.938347 12463 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:18:02.939185 12481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:02.939356 12463 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:02.939488 12463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "5957ec96398e422b94e2b958ce4c5099"
format_stamp: "Formatted at 2026-08-12 14:18:02 on dist-test-slave-5k54"
server_key: "1d59b1934eb595f4019cebe4fdda88c4"
server_key_iv: "63d858e58843dafd9d2d905d561961bb"
server_key_version: "encryptionkey@0"
I20260812 14:18:02.939569 12463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:02.956308 12463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:02.958150 12463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:02.958319 12463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:02.958551 12463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:02.958886 12463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:02.958946 12463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:02.958988 12463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:02.959025 12463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:02.965368 12463 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:42111
I20260812 14:18:02.965412 12615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:42111 every 8 connection(s)
I20260812 14:18:02.965732 12463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:18:02.970237 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 12463
I20260812 14:18:02.970325 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:18:02.970535 8532 external_mini_cluster.cc:1428] Setting key 37739bb9649fbfde2bb6c1ced7f0a2ee
Aug 12 14:18:02 dist-test-slave-5k54 krb5kdc[12035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544282, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:02.974788 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:02.966403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:54229 (local address 127.8.85.62:36181)
0812 14:18:02.966468 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:02.966471 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:02.966699 (+ 228us) server_negotiation.cc:407] Connection header received
0812 14:18:02.967170 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:02.967171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:02.967182 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:02.967235 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:02.968218 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:02.968707 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:02.969638 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:02.969839 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:02.969890 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:02.972262 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:02.972263 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:02.972263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:02.972270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:02.973698 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:02.974037 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:02.974038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:02.974038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:02.974099 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:02.974298 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:02.974299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:02.974299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:02.974394 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:02.974443 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:02.974669 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 14:18:02.974727 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:18:02.974942 12618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:02.966382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:36181 (local address 127.8.85.3:54229)
0812 14:18:02.966582 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:02.966618 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:02.967091 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:02.967303 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:02.967306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:02.967950 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:02.968141 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:02.968146 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:02.968787 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:02.968788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:02.969568 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:02.969570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:02.969912 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:02.969913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:02.970013 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:02.970594 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:02.970605 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:02.972185 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:02.973766 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:02.973772 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:02.973773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:02.973975 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:02.974155 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:02.974155 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:02.974156 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:02.974248 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:02.974446 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:02.974451 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:02.974567 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 14:18:02.974645 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 14:18:02.974756 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":106,"threads_started":1}
I20260812 14:18:02.975380 12616 heartbeater.cc:347] Connected to a master server at 127.8.85.62:36181
I20260812 14:18:02.975513 12616 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:02.975689 12616 heartbeater.cc:511] Master 127.8.85.62:36181 requested a full tablet report, sending...
I20260812 14:18:02.976104 12086 ts_manager.cc:194] Registered new tserver with Master: 5957ec96398e422b94e2b958ce4c5099 (127.8.85.3:42111)
I20260812 14:18:02.976475 12086 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:54229
I20260812 14:18:02.982388 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 14:18:02 dist-test-slave-5k54 krb5kdc[12035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544282, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 14:18:03 dist-test-slave-5k54 krb5kdc[12035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544282, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:03.007189 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:02.999802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59726 (local address 127.8.85.62:36181)
0812 14:18:02.999862 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:02.999864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:03.000006 (+ 142us) server_negotiation.cc:407] Connection header received
0812 14:18:03.000091 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:03.000092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:03.000104 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:03.000164 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:03.000791 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:03.001299 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:03.002106 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:03.002314 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:03.002391 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:03.004690 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:03.004692 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:03.004692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:03.004700 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:03.005957 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:03.006309 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:03.006310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:03.006311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:03.006373 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:03.006584 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:03.006585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:03.006586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:03.006741 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:03.006845 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:03.006992 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 14:18:03.007082 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260812 14:18:03.007459 12086 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59726
I20260812 14:18:03.008111 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 12158
I20260812 14:18:03.018587 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 12312
I20260812 14:18:03.026042 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 12463
I20260812 14:18:03.033326 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 12051
2026-08-12T14:18:03Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2004 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:18:03 dist-test-slave-5k54 krb5kdc[12629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:18:03 dist-test-slave-5k54 krb5kdc[12629](info): set up 2 sockets
Aug 12 14:18:03 dist-test-slave-5k54 krb5kdc[12629](info): commencing operation
krb5kdc: starting...
W20260812 14:18:04.153494 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:18:04 dist-test-slave-5k54 krb5kdc[12629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:18:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:18:04.279265 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:39027
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:37509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:39027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:04.366602 12643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:04.366837 12643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:04.366878 12643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:04.368050 12643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:04.368125 12643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:04.368149 12643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:04.368171 12643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:04.368201 12643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:04.369812 12643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:37509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:39027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:39027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:04.370096 12643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:04.370340 12643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:04.375315 12653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:04.375324 12650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:04.375542 12643 server_base.cc:1034] running on GCE node
W20260812 14:18:04.375635 12651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:04.375906 12643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:04.376209 12643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:04.377408 12643 hybrid_clock.cc:648] HybridClock initialized: now 1786544284377385 us; error 43 us; skew 500 ppm
Aug 12 14:18:04 dist-test-slave-5k54 krb5kdc[12629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:04.380051 12643 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:18:04.380494 12643 webserver.cc:460] Webserver started at http://127.8.85.62:41909/ using document root <none> and password file <none>
I20260812 14:18:04.380743 12643 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:04.380791 12643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:04.380951 12643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:04.382035 12643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "6993b9095f784364a1e515a1776f1452"
format_stamp: "Formatted at 2026-08-12 14:18:04 on dist-test-slave-5k54"
server_key: "b222ca5b1ac92af28c7e06ba49a42570"
server_key_iv: "8e18e43e17135b542e309c5a35b15833"
server_key_version: "encryptionkey@0"
I20260812 14:18:04.382414 12643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "6993b9095f784364a1e515a1776f1452"
format_stamp: "Formatted at 2026-08-12 14:18:04 on dist-test-slave-5k54"
server_key: "b222ca5b1ac92af28c7e06ba49a42570"
server_key_iv: "8e18e43e17135b542e309c5a35b15833"
server_key_version: "encryptionkey@0"
I20260812 14:18:04.383773 12643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:18:04.384480 12661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:04.384655 12643 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:04.384747 12643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "6993b9095f784364a1e515a1776f1452"
format_stamp: "Formatted at 2026-08-12 14:18:04 on dist-test-slave-5k54"
server_key: "b222ca5b1ac92af28c7e06ba49a42570"
server_key_iv: "8e18e43e17135b542e309c5a35b15833"
server_key_version: "encryptionkey@0"
I20260812 14:18:04.384835 12643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:04.394105 12643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:04.395895 12643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:04.396045 12643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:04.399904 12643 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:39027
I20260812 14:18:04.399935 12725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:39027 every 8 connection(s)
I20260812 14:18:04.400489 12643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:04.401137 12726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:04.403349 12726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452: Bootstrap starting.
I20260812 14:18:04.403990 12726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:04.403983 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 12643
I20260812 14:18:04.404101 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:04.404264 8532 external_mini_cluster.cc:1428] Setting key 9808e07130e300d8a6542c90638e0f5a
I20260812 14:18:04.404306 12726 log.cc:824] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:04.405159 12726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452: No bootstrap required, opened a new log
I20260812 14:18:04.407622 12726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6993b9095f784364a1e515a1776f1452" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 39027 } }
I20260812 14:18:04.407793 12726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:04.407828 12726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6993b9095f784364a1e515a1776f1452, State: Initialized, Role: FOLLOWER
I20260812 14:18:04.407961 12726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6993b9095f784364a1e515a1776f1452" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 39027 } }
I20260812 14:18:04.408030 12726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:04.408058 12726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:04.408099 12726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:04.408712 12726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6993b9095f784364a1e515a1776f1452" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 39027 } }
I20260812 14:18:04.408836 12726 leader_election.cc:302] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6993b9095f784364a1e515a1776f1452; no voters:
I20260812 14:18:04.409029 12726 leader_election.cc:288] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:04.409065 12731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:04.409296 12726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:04.409554 12731 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [term 1 LEADER]: Becoming Leader. State: Replica: 6993b9095f784364a1e515a1776f1452, State: Running, Role: LEADER
I20260812 14:18:04.409683 12731 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6993b9095f784364a1e515a1776f1452" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 39027 } }
I20260812 14:18:04.410032 12732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6993b9095f784364a1e515a1776f1452" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6993b9095f784364a1e515a1776f1452" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 39027 } } }
I20260812 14:18:04.410109 12732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:04.410279 12733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6993b9095f784364a1e515a1776f1452. Latest consensus state: current_term: 1 leader_uuid: "6993b9095f784364a1e515a1776f1452" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6993b9095f784364a1e515a1776f1452" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 39027 } } }
I20260812 14:18:04.410331 12733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:04.410597 12742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:04.411134 12742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 14:18:04 dist-test-slave-5k54 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:04.412895 12742 catalog_manager.cc:1347] Generated new cluster ID: d160ec4aad12403fbfe4065c9297fe7f
I20260812 14:18:04.412986 12742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:18:04.415679 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:04.404764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59610 (local address 127.8.85.62:39027)
0812 14:18:04.404920 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:04.404928 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:04.404973 (+ 45us) server_negotiation.cc:407] Connection header received
0812 14:18:04.405456 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:04.405463 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:04.405510 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:04.405622 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:04.406421 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.407264 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.408359 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.408753 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.408878 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:04.411948 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:04.411952 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:04.411953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:04.411964 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:04.413970 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:04.414353 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:04.414362 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:04.414363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:04.414471 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:04.414679 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:04.414680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:04.414680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:04.414988 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:04.415170 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:04.415248 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 14:18:04.415414 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":75,"threads_started":1}
I20260812 14:18:04.430020 12742 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:04.430542 12742 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:04.436803 12742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6993b9095f784364a1e515a1776f1452: Generated new TSK 0
I20260812 14:18:04.437012 12742 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:18:04.467183 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:39027
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:37509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:04.551872 12758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:04.552068 12758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:04.552091 12758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:04.553256 12758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:04.553297 12758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:04.553332 12758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:18:04.554839 12758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:37509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:39027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:04.555064 12758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:04.555311 12758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:04.560210 12764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:04.560235 12763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:04.560210 12767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:04.560432 12758 server_base.cc:1034] running on GCE node
I20260812 14:18:04.560655 12758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:04.560896 12758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:04.562089 12758 hybrid_clock.cc:648] HybridClock initialized: now 1786544284562076 us; error 27 us; skew 500 ppm
Aug 12 14:18:04 dist-test-slave-5k54 krb5kdc[12629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:04.570526 12758 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:18:04.571029 12758 webserver.cc:460] Webserver started at http://127.8.85.1:33229/ using document root <none> and password file <none>
I20260812 14:18:04.571259 12758 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:04.571328 12758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:04.571511 12758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:04.572561 12758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "59ed4c447c324d0188d0c840258d96a8"
format_stamp: "Formatted at 2026-08-12 14:18:04 on dist-test-slave-5k54"
server_key: "d3f41318b19b04414ec026970d133e79"
server_key_iv: "a9b9cf5f9fe924e515f181530d6b0f7f"
server_key_version: "encryptionkey@0"
I20260812 14:18:04.572929 12758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "59ed4c447c324d0188d0c840258d96a8"
format_stamp: "Formatted at 2026-08-12 14:18:04 on dist-test-slave-5k54"
server_key: "d3f41318b19b04414ec026970d133e79"
server_key_iv: "a9b9cf5f9fe924e515f181530d6b0f7f"
server_key_version: "encryptionkey@0"
I20260812 14:18:04.574250 12758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:18:04.574981 12775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:04.575171 12758 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:04.575278 12758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "59ed4c447c324d0188d0c840258d96a8"
format_stamp: "Formatted at 2026-08-12 14:18:04 on dist-test-slave-5k54"
server_key: "d3f41318b19b04414ec026970d133e79"
server_key_iv: "a9b9cf5f9fe924e515f181530d6b0f7f"
server_key_version: "encryptionkey@0"
I20260812 14:18:04.575350 12758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:04.586728 12758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:04.589083 12758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:04.589192 12758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:04.589359 12758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:04.589677 12758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:04.589715 12758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:04.589736 12758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:04.589780 12758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:04.595741 12758 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:36995
I20260812 14:18:04.595793 12914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:36995 every 8 connection(s)
I20260812 14:18:04.596141 12758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:18:04 dist-test-slave-5k54 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:04.602970 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 12758
I20260812 14:18:04.603091 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:18:04.603309 8532 external_mini_cluster.cc:1428] Setting key f9de39329bb12e6b64ea0cbd27391453
I20260812 14:18:04.605710 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:04.596948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:51301 (local address 127.8.85.62:39027)
0812 14:18:04.597053 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:04.597055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:04.597230 (+ 175us) server_negotiation.cc:407] Connection header received
0812 14:18:04.598181 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:04.598183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:04.598195 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:04.598249 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:04.598857 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.599392 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.600435 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.600643 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.600719 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:04.602738 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:04.602740 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:04.602740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:04.602748 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:04.604307 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:04.604718 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:04.604719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:04.604719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:04.604795 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:04.605065 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:04.605066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:04.605066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:04.605180 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:04.605248 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:04.605530 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 14:18:04.605621 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 14:18:04.605907 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:04.596864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:39027 (local address 127.8.85.1:51301)
0812 14:18:04.597084 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:04.597135 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:04.598066 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:04.598319 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:04.598323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:04.598578 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:04.598754 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:04.598757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.599533 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.599534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:04.600330 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:04.600332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.600726 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.600728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:04.600866 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:04.601289 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:04.601298 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:04.602638 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:04.604414 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:04.604422 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:04.604424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:04.604640 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:04.604876 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:04.604877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:04.604877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:04.604984 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:04.605252 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:04.605259 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:04.605394 (+ 135us) client_negotiation.cc:769] Sending connection context
0812 14:18:04.605496 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 14:18:04.605648 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":84,"threads_started":1}
I20260812 14:18:04.606443 12915 heartbeater.cc:347] Connected to a master server at 127.8.85.62:39027
I20260812 14:18:04.606536 12915 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:04.606724 12915 heartbeater.cc:511] Master 127.8.85.62:39027 requested a full tablet report, sending...
I20260812 14:18:04.607228 12683 ts_manager.cc:194] Registered new tserver with Master: 59ed4c447c324d0188d0c840258d96a8 (127.8.85.1:36995)
I20260812 14:18:04.607875 12683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:51301
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:18:04.630137 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:39027
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:37509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:04.711859 12920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:04.712086 12920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:04.712112 12920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:04.713258 12920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:04.713301 12920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:04.713328 12920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:18:04.714830 12920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:37509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:39027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:04.715071 12920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:04.715318 12920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:04.720443 12929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:04.720451 12926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:04.720636 12920 server_base.cc:1034] running on GCE node
W20260812 14:18:04.720486 12927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:04.720927 12920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:04.721181 12920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:04.722329 12920 hybrid_clock.cc:648] HybridClock initialized: now 1786544284722319 us; error 39 us; skew 500 ppm
Aug 12 14:18:04 dist-test-slave-5k54 krb5kdc[12629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:04.725063 12920 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:18:04.725534 12920 webserver.cc:460] Webserver started at http://127.8.85.2:36385/ using document root <none> and password file <none>
I20260812 14:18:04.725767 12920 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:04.725833 12920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:04.725970 12920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:04.727030 12920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "9eb4e837a9424bb78433307ffd7c17eb"
format_stamp: "Formatted at 2026-08-12 14:18:04 on dist-test-slave-5k54"
server_key: "aeeb0d71943a41133606053ffc4a7a56"
server_key_iv: "1ad05456072ffc5012a40ab09b4f70e1"
server_key_version: "encryptionkey@0"
I20260812 14:18:04.727398 12920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "9eb4e837a9424bb78433307ffd7c17eb"
format_stamp: "Formatted at 2026-08-12 14:18:04 on dist-test-slave-5k54"
server_key: "aeeb0d71943a41133606053ffc4a7a56"
server_key_iv: "1ad05456072ffc5012a40ab09b4f70e1"
server_key_version: "encryptionkey@0"
I20260812 14:18:04.728813 12920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:04.729548 12938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:04.729722 12920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:04.729820 12920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "9eb4e837a9424bb78433307ffd7c17eb"
format_stamp: "Formatted at 2026-08-12 14:18:04 on dist-test-slave-5k54"
server_key: "aeeb0d71943a41133606053ffc4a7a56"
server_key_iv: "1ad05456072ffc5012a40ab09b4f70e1"
server_key_version: "encryptionkey@0"
I20260812 14:18:04.729898 12920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:04.738497 12920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:04.740492 12920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:04.740653 12920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:04.740867 12920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:04.741183 12920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:04.741235 12920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:04.741268 12920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:04.741297 12920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:04.747320 12920 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:42335
I20260812 14:18:04.747366 13075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:42335 every 8 connection(s)
I20260812 14:18:04.747694 12920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:18:04 dist-test-slave-5k54 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:04.754966 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 12920
I20260812 14:18:04.755101 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:18:04.755299 8532 external_mini_cluster.cc:1428] Setting key 84c1275bbe106b391c2c2f15d660507c
I20260812 14:18:04.757150 13079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:04.748397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:39027 (local address 127.8.85.2:47051)
0812 14:18:04.748572 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:04.748607 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:04.749476 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:04.749690 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:04.749692 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:04.749922 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:04.750100 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:04.750104 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.750771 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.750772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:04.751570 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:04.751572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.751923 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.751923 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:04.752018 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:04.752426 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:04.752434 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:04.753811 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:04.755565 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:04.755572 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:04.755574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:04.755867 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:04.756113 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:04.756114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:04.756114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:04.756241 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:04.756543 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:04.756547 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:04.756699 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 14:18:04.756871 (+ 172us) client_negotiation.cc:240] Negotiation successful
0812 14:18:04.756994 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260812 14:18:04.757146 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:04.748418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:47051 (local address 127.8.85.62:39027)
0812 14:18:04.748487 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:04.748490 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:04.748721 (+ 231us) server_negotiation.cc:407] Connection header received
0812 14:18:04.749573 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:04.749574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:04.749585 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:04.749639 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:04.750188 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.750682 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.751643 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.751852 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.751904 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:04.753914 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:04.753915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:04.753916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:04.753924 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:04.755474 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:04.755971 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:04.755972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:04.755973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:04.756038 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:04.756323 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:04.756324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:04.756325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:04.756472 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:04.756547 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:04.756908 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 14:18:04.757009 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:18:04.757793 13077 heartbeater.cc:347] Connected to a master server at 127.8.85.62:39027
I20260812 14:18:04.757896 13077 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:04.758106 13077 heartbeater.cc:511] Master 127.8.85.62:39027 requested a full tablet report, sending...
I20260812 14:18:04.758548 12683 ts_manager.cc:194] Registered new tserver with Master: 9eb4e837a9424bb78433307ffd7c17eb (127.8.85.2:42335)
I20260812 14:18:04.759045 12683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:47051
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:18:04.782703 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:39027
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:37509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:04.864552 13082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:04.864746 13082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:04.864785 13082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:04.865934 13082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:04.865976 13082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:04.866036 13082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:18:04.867551 13082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:37509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:39027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:04.867777 13082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:04.868001 13082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:04.874467 13089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:04.874727 13092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:04.874716 13082 server_base.cc:1034] running on GCE node
W20260812 14:18:04.874495 13088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:04.875079 13082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:04.875335 13082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:04.876498 13082 hybrid_clock.cc:648] HybridClock initialized: now 1786544284876485 us; error 27 us; skew 500 ppm
Aug 12 14:18:04 dist-test-slave-5k54 krb5kdc[12629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:04.879887 13082 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:18:04.880281 13082 webserver.cc:460] Webserver started at http://127.8.85.3:33247/ using document root <none> and password file <none>
I20260812 14:18:04.880461 13082 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:04.880501 13082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:04.880596 13082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:04.881613 13082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "9481befc90eb4067b824639de0066c2b"
format_stamp: "Formatted at 2026-08-12 14:18:04 on dist-test-slave-5k54"
server_key: "2b458d0647f4f7fea449a3c52fa4d58f"
server_key_iv: "285d5faff35f502d2fc65a5e8ae3d29e"
server_key_version: "encryptionkey@0"
I20260812 14:18:04.881923 13082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "9481befc90eb4067b824639de0066c2b"
format_stamp: "Formatted at 2026-08-12 14:18:04 on dist-test-slave-5k54"
server_key: "2b458d0647f4f7fea449a3c52fa4d58f"
server_key_iv: "285d5faff35f502d2fc65a5e8ae3d29e"
server_key_version: "encryptionkey@0"
I20260812 14:18:04.883186 13082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:18:04.883952 13100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:04.884132 13082 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:04.884233 13082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "9481befc90eb4067b824639de0066c2b"
format_stamp: "Formatted at 2026-08-12 14:18:04 on dist-test-slave-5k54"
server_key: "2b458d0647f4f7fea449a3c52fa4d58f"
server_key_iv: "285d5faff35f502d2fc65a5e8ae3d29e"
server_key_version: "encryptionkey@0"
I20260812 14:18:04.884322 13082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:04.896710 13082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:04.898639 13082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:04.898806 13082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:04.899049 13082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:04.899354 13082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:04.899408 13082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:04.899456 13082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:04.899506 13082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:04.905762 13082 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:39495
I20260812 14:18:04.905817 13237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:39495 every 8 connection(s)
I20260812 14:18:04.906131 13082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:18:04.907773 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 13082
I20260812 14:18:04.907893 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:18:04.908170 8532 external_mini_cluster.cc:1428] Setting key 016fa72c6ddeddd48e6389ef058effa5
Aug 12 14:18:04 dist-test-slave-5k54 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544284, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:04.916397 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:04.906958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:34831 (local address 127.8.85.62:39027)
0812 14:18:04.907023 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:04.907025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:04.907225 (+ 200us) server_negotiation.cc:407] Connection header received
0812 14:18:04.908596 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:04.908597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:04.908611 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:04.908685 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:04.909515 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.910068 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.911320 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.911561 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.911619 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:04.913755 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:04.913756 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:04.913757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:04.913764 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:04.914984 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:04.915333 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:04.915334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:04.915335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:04.915397 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:04.915860 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:04.915861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:04.915861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:04.915972 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:04.916025 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:04.916257 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 14:18:04.916321 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:18:04.916587 13241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:04.906936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:39027 (local address 127.8.85.3:34831)
0812 14:18:04.907105 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:04.907141 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:04.908419 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:04.908775 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:04.908779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:04.909216 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:04.909434 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:04.909440 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.910144 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.910145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:04.911244 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:04.911246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.911644 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.911645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:04.911780 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:04.912255 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:04.912263 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:04.913672 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:04.915054 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:04.915060 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:04.915062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:04.915270 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:04.915694 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:04.915695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:04.915695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:04.915796 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:04.916034 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:04.916038 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:04.916148 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 14:18:04.916228 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 14:18:04.916346 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260812 14:18:04.917132 13238 heartbeater.cc:347] Connected to a master server at 127.8.85.62:39027
I20260812 14:18:04.917245 13238 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:04.917428 13238 heartbeater.cc:511] Master 127.8.85.62:39027 requested a full tablet report, sending...
I20260812 14:18:04.917814 12683 ts_manager.cc:194] Registered new tserver with Master: 9481befc90eb4067b824639de0066c2b (127.8.85.3:39495)
I20260812 14:18:04.918200 12683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:34831
I20260812 14:18:04.919950 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 14:18:04 dist-test-slave-5k54 krb5kdc[12629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 14:18:04 dist-test-slave-5k54 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.1@KRBTEST.COM
I20260812 14:18:04.943873 13246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:04.936067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35924 (local address 127.8.85.1:36995)
0812 14:18:04.936227 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:04.936232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:04.936242 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:18:04.936267 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:04.936269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:04.936283 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:04.936350 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:04.936968 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.937714 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.938471 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.938805 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.938888 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:04.941200 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:04.941209 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:04.941209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:04.941217 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:04.942603 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:04.942969 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:04.942973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:04.942973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:04.943038 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:04.943246 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:04.943247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:04.943247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:04.943392 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:04.943582 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:04.943668 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 14:18:04.943749 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
Aug 12 14:18:04 dist-test-slave-5k54 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:04.951916 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:04.945254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59624 (local address 127.8.85.62:39027)
0812 14:18:04.945325 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:04.945328 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:04.945337 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:18:04.945390 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:04.945392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:04.945402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:04.945458 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:04.945951 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.946451 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.947130 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:04.947326 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:04.947402 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:04.949699 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:04.949700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:04.949701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:04.949707 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:04.950876 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:04.951171 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:04.951172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:04.951172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:04.951223 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:04.951380 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:04.951381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:04.951381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:04.951620 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:04.951700 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:04.951787 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 14:18:04.951850 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 14:18:04.952180 12683 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59624
I20260812 14:18:04.952569 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 12758
I20260812 14:18:04.968139 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 12920
I20260812 14:18:04.974975 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 13082
I20260812 14:18:04.985634 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 12643
2026-08-12T14:18:04Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1953 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:18:05 dist-test-slave-5k54 krb5kdc[13251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:18:05 dist-test-slave-5k54 krb5kdc[13251](info): set up 2 sockets
Aug 12 14:18:05 dist-test-slave-5k54 krb5kdc[13251](info): commencing operation
krb5kdc: starting...
W20260812 14:18:06.119324 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:18:06 dist-test-slave-5k54 krb5kdc[13251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:18:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:18:06.246467 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:33061
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:33061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:06.328675 13264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:06.328894 13264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:06.328918 13264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:06.330017 13264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:06.330053 13264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:06.330068 13264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:06.330080 13264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:06.330092 13264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:06.331574 13264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:33061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:33061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:06.331774 13264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:06.331943 13264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:06.336751 13272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:06.336724 13271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:06.336727 13274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:06.336812 13264 server_base.cc:1034] running on GCE node
I20260812 14:18:06.337283 13264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:06.337548 13264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:06.338699 13264 hybrid_clock.cc:648] HybridClock initialized: now 1786544286338692 us; error 32 us; skew 500 ppm
Aug 12 14:18:06 dist-test-slave-5k54 krb5kdc[13251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544286, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:06.341239 13264 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:18:06.341637 13264 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:06.341688 13264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:06.341861 13264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:06.342940 13264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "0261eaf5a528441da33bf6883f9da222"
format_stamp: "Formatted at 2026-08-12 14:18:06 on dist-test-slave-5k54"
server_key: "98faab391aaa260d3d777c6c18791041"
server_key_iv: "4c252bce46d65d5ab0c4ece82b4158d6"
server_key_version: "encryptionkey@0"
I20260812 14:18:06.343303 13264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "0261eaf5a528441da33bf6883f9da222"
format_stamp: "Formatted at 2026-08-12 14:18:06 on dist-test-slave-5k54"
server_key: "98faab391aaa260d3d777c6c18791041"
server_key_iv: "4c252bce46d65d5ab0c4ece82b4158d6"
server_key_version: "encryptionkey@0"
I20260812 14:18:06.344642 13264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:18:06.345387 13282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:06.345582 13264 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:06.345677 13264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "0261eaf5a528441da33bf6883f9da222"
format_stamp: "Formatted at 2026-08-12 14:18:06 on dist-test-slave-5k54"
server_key: "98faab391aaa260d3d777c6c18791041"
server_key_iv: "4c252bce46d65d5ab0c4ece82b4158d6"
server_key_version: "encryptionkey@0"
I20260812 14:18:06.345752 13264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:06.369096 13264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:06.371018 13264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:06.371183 13264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:06.375312 13264 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:33061
I20260812 14:18:06.375342 13344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:33061 every 8 connection(s)
I20260812 14:18:06.375651 13264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:06.376704 13345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:06.379052 13345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222: Bootstrap starting.
I20260812 14:18:06.379698 13345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:06.380002 13345 log.cc:824] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:06.380702 13345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222: No bootstrap required, opened a new log
I20260812 14:18:06.381592 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 13264
I20260812 14:18:06.381675 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:06.381855 8532 external_mini_cluster.cc:1428] Setting key b2d0811330800c27175d564632533a6b
I20260812 14:18:06.382558 13345 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0261eaf5a528441da33bf6883f9da222" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33061 } }
I20260812 14:18:06.382733 13345 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:06.382784 13345 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0261eaf5a528441da33bf6883f9da222, State: Initialized, Role: FOLLOWER
I20260812 14:18:06.382951 13345 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0261eaf5a528441da33bf6883f9da222" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33061 } }
I20260812 14:18:06.383049 13345 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:06.383108 13345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:06.383168 13345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:06.383796 13345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0261eaf5a528441da33bf6883f9da222" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33061 } }
I20260812 14:18:06.383919 13345 leader_election.cc:302] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0261eaf5a528441da33bf6883f9da222; no voters:
I20260812 14:18:06.384110 13345 leader_election.cc:288] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:06.384145 13352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:06.384255 13352 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [term 1 LEADER]: Becoming Leader. State: Replica: 0261eaf5a528441da33bf6883f9da222, State: Running, Role: LEADER
I20260812 14:18:06.384338 13352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0261eaf5a528441da33bf6883f9da222" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33061 } }
I20260812 14:18:06.384394 13345 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:06.384668 13352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0261eaf5a528441da33bf6883f9da222" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0261eaf5a528441da33bf6883f9da222" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33061 } } }
I20260812 14:18:06.384850 13352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:06.385056 13352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0261eaf5a528441da33bf6883f9da222. Latest consensus state: current_term: 1 leader_uuid: "0261eaf5a528441da33bf6883f9da222" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0261eaf5a528441da33bf6883f9da222" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33061 } } }
I20260812 14:18:06.385135 13352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:06.385519 13362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:06.386178 13362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:06.387744 13362 catalog_manager.cc:1347] Generated new cluster ID: 23f15b5cee9946289d40df53fd545a5b
I20260812 14:18:06.387801 13362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:18:06 dist-test-slave-5k54 krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:06.392918 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:06.382442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46208 (local address 127.8.85.62:33061)
0812 14:18:06.382640 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:06.382648 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:06.382674 (+ 26us) server_negotiation.cc:407] Connection header received
0812 14:18:06.382792 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:06.382799 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:06.382863 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:06.383027 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:06.383929 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.385358 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.386526 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.387142 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.387249 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:06.389652 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:06.389656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:06.389656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:06.389674 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:06.391355 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:06.391785 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:06.391788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:06.391788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:06.391890 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:06.392124 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:06.392125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:06.392125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:06.392361 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:06.392547 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:06.392616 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 14:18:06.392757 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260812 14:18:06.400650 13362 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:06.401374 13362 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:06.412339 13362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222: Generated new TSK 0
I20260812 14:18:06.412578 13362 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:18:06.440536 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33061
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:06.524453 13378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:06.524693 13378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:06.524731 13378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:06.525936 13378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:06.526005 13378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:06.526052 13378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:18:06.527695 13378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:06.527973 13378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:06.528213 13378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:06.533212 13384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:06.533284 13387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:06.533192 13385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:06.533489 13378 server_base.cc:1034] running on GCE node
I20260812 14:18:06.533700 13378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:06.533921 13378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:06.535077 13378 hybrid_clock.cc:648] HybridClock initialized: now 1786544286535061 us; error 31 us; skew 500 ppm
Aug 12 14:18:06 dist-test-slave-5k54 krb5kdc[13251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544286, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:06.537746 13378 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:18:06.538172 13378 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:06.538245 13378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:06.538399 13378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:06.539541 13378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "5d45f95cece14033a51dbccf68fbc5b4"
format_stamp: "Formatted at 2026-08-12 14:18:06 on dist-test-slave-5k54"
server_key: "bbdb2fd86d1009594df307c8fbc7895f"
server_key_iv: "878a9629c7ed78bff7372eb836a889c8"
server_key_version: "encryptionkey@0"
I20260812 14:18:06.539909 13378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "5d45f95cece14033a51dbccf68fbc5b4"
format_stamp: "Formatted at 2026-08-12 14:18:06 on dist-test-slave-5k54"
server_key: "bbdb2fd86d1009594df307c8fbc7895f"
server_key_iv: "878a9629c7ed78bff7372eb836a889c8"
server_key_version: "encryptionkey@0"
I20260812 14:18:06.541198 13378 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:06.541924 13394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:06.542090 13378 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:06.542183 13378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "5d45f95cece14033a51dbccf68fbc5b4"
format_stamp: "Formatted at 2026-08-12 14:18:06 on dist-test-slave-5k54"
server_key: "bbdb2fd86d1009594df307c8fbc7895f"
server_key_iv: "878a9629c7ed78bff7372eb836a889c8"
server_key_version: "encryptionkey@0"
I20260812 14:18:06.542265 13378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:06.565971 13378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:06.567901 13378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:06.568070 13378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:06.568329 13378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:06.568665 13378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:06.568722 13378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:06.568764 13378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:06.568794 13378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:06.574910 13378 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:33717
I20260812 14:18:06.574985 13532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:33717 every 8 connection(s)
I20260812 14:18:06.575260 13378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:18:06.576457 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 13378
I20260812 14:18:06.576573 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:18:06.576756 8532 external_mini_cluster.cc:1428] Setting key 91f105f2473a237367d92de2d1eda375
Aug 12 14:18:06 dist-test-slave-5k54 krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544286, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:06.586472 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:06.576066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:39195 (local address 127.8.85.62:33061)
0812 14:18:06.576122 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:06.576124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:06.576327 (+ 203us) server_negotiation.cc:407] Connection header received
0812 14:18:06.576931 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:06.576933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:06.576945 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:06.577000 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:06.578292 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.579051 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.580686 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.581013 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.581091 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:06.583212 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:06.583214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:06.583214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:06.583224 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:06.584927 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:06.585394 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:06.585395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:06.585396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:06.585455 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:06.585691 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:06.585692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:06.585692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:06.585845 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:06.585928 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:06.586266 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 14:18:06.586362 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:18:06.586549 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:06.576050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33061 (local address 127.8.85.1:39195)
0812 14:18:06.576190 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:06.576226 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:06.576790 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:06.577111 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:06.577116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:06.577491 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:06.578182 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:06.578188 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.579160 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.579161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:06.580574 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:06.580577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.581100 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.581101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:06.581199 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:06.581732 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:06.581741 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:06.583120 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:06.585020 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:06.585027 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:06.585029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:06.585319 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:06.585527 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:06.585527 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:06.585528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:06.585625 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:06.585919 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:06.585923 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:06.586087 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 14:18:06.586222 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 14:18:06.586360 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
I20260812 14:18:06.587105 13533 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33061
I20260812 14:18:06.587222 13533 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:06.587369 13533 heartbeater.cc:511] Master 127.8.85.62:33061 requested a full tablet report, sending...
I20260812 14:18:06.588047 13302 ts_manager.cc:194] Registered new tserver with Master: 5d45f95cece14033a51dbccf68fbc5b4 (127.8.85.1:33717)
I20260812 14:18:06.588675 13302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:39195
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:18:06.606149 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33061
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:06.689944 13539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:06.690152 13539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:06.690176 13539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:06.691337 13539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:06.691380 13539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:06.691406 13539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:18:06.692885 13539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:06.693074 13539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:06.693250 13539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:06.698071 13549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:06.698093 13546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:06.698092 13547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:06.698469 13539 server_base.cc:1034] running on GCE node
I20260812 14:18:06.698680 13539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:06.698925 13539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:06.700083 13539 hybrid_clock.cc:648] HybridClock initialized: now 1786544286700081 us; error 36 us; skew 500 ppm
Aug 12 14:18:06 dist-test-slave-5k54 krb5kdc[13251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544286, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:06.702538 13539 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:18:06.702930 13539 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:06.703003 13539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:06.703162 13539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:06.704259 13539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "54c66c3be05a4522a9603da6455755d2"
format_stamp: "Formatted at 2026-08-12 14:18:06 on dist-test-slave-5k54"
server_key: "5552bf55292e9ffe9127f4670b5776b0"
server_key_iv: "d458241ba0494829410a5cee0928dd59"
server_key_version: "encryptionkey@0"
I20260812 14:18:06.704618 13539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "54c66c3be05a4522a9603da6455755d2"
format_stamp: "Formatted at 2026-08-12 14:18:06 on dist-test-slave-5k54"
server_key: "5552bf55292e9ffe9127f4670b5776b0"
server_key_iv: "d458241ba0494829410a5cee0928dd59"
server_key_version: "encryptionkey@0"
I20260812 14:18:06.705943 13539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:18:06.706636 13555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:06.706825 13539 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:06.706954 13539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "54c66c3be05a4522a9603da6455755d2"
format_stamp: "Formatted at 2026-08-12 14:18:06 on dist-test-slave-5k54"
server_key: "5552bf55292e9ffe9127f4670b5776b0"
server_key_iv: "d458241ba0494829410a5cee0928dd59"
server_key_version: "encryptionkey@0"
I20260812 14:18:06.707028 13539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:06.716464 13539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:06.718225 13539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:06.718384 13539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:06.718611 13539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:06.718925 13539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:06.718978 13539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:06.719019 13539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:06.719048 13539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:06.724805 13539 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:40103
I20260812 14:18:06.724843 13687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:40103 every 8 connection(s)
I20260812 14:18:06.725118 13539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:18:06 dist-test-slave-5k54 krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544286, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:06.731554 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 13539
I20260812 14:18:06.731671 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:18:06.731907 8532 external_mini_cluster.cc:1428] Setting key 7f78957f0304b5d4bb0dde4d217d5c9a
I20260812 14:18:06.734360 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:06.725820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:33415 (local address 127.8.85.62:33061)
0812 14:18:06.725884 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:06.725886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:06.726101 (+ 215us) server_negotiation.cc:407] Connection header received
0812 14:18:06.726570 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:06.726571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:06.726582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:06.726641 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:06.727619 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.728180 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.729131 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.729322 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.729391 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:06.731520 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:06.731521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:06.731522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:06.731535 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:06.733018 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:06.733431 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:06.733432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:06.733432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:06.733493 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:06.733736 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:06.733737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:06.733738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:06.733849 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:06.733924 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:06.734188 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 14:18:06.734272 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:18:06.734588 13691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:06.725781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33061 (local address 127.8.85.2:33415)
0812 14:18:06.725966 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:06.726007 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:06.726487 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:06.726710 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:06.726713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:06.726975 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:06.727535 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:06.727541 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.728259 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.728260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:06.729056 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:06.729058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.729403 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.729403 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:06.729504 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:06.730021 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:06.730030 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:06.731374 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:06.733111 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:06.733118 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:06.733121 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:06.733354 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:06.733558 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:06.733559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:06.733559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:06.733670 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:06.733914 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:06.733920 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:06.734060 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 14:18:06.734168 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 14:18:06.734316 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260812 14:18:06.734961 13688 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33061
I20260812 14:18:06.735081 13688 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:06.735242 13688 heartbeater.cc:511] Master 127.8.85.62:33061 requested a full tablet report, sending...
I20260812 14:18:06.735612 13302 ts_manager.cc:194] Registered new tserver with Master: 54c66c3be05a4522a9603da6455755d2 (127.8.85.2:40103)
I20260812 14:18:06.735944 13302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:33415
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:18:06.759137 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33061
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:06.842561 13695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:06.842767 13695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:06.842790 13695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:06.843930 13695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:06.843976 13695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:06.844002 13695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:18:06.845638 13695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:06.845868 13695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:06.846061 13695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:06.851382 13701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:06.851385 13700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:06.851393 13704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:06.851780 13695 server_base.cc:1034] running on GCE node
I20260812 14:18:06.851967 13695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:06.852154 13695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:06.853335 13695 hybrid_clock.cc:648] HybridClock initialized: now 1786544286853323 us; error 40 us; skew 500 ppm
Aug 12 14:18:06 dist-test-slave-5k54 krb5kdc[13251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544286, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:06.856004 13695 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:18:06.856407 13695 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:06.856482 13695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:06.856632 13695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:06.857777 13695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "731d5c31effa44049df1aa8623159a2c"
format_stamp: "Formatted at 2026-08-12 14:18:06 on dist-test-slave-5k54"
server_key: "409c0595a34c67ea7d2fb9e15cdb14f4"
server_key_iv: "6188393bcf452a7093a6feb5d8d9256a"
server_key_version: "encryptionkey@0"
I20260812 14:18:06.858160 13695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "731d5c31effa44049df1aa8623159a2c"
format_stamp: "Formatted at 2026-08-12 14:18:06 on dist-test-slave-5k54"
server_key: "409c0595a34c67ea7d2fb9e15cdb14f4"
server_key_iv: "6188393bcf452a7093a6feb5d8d9256a"
server_key_version: "encryptionkey@0"
I20260812 14:18:06.859565 13695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:18:06.860330 13710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:06.860528 13695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:06.860630 13695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "731d5c31effa44049df1aa8623159a2c"
format_stamp: "Formatted at 2026-08-12 14:18:06 on dist-test-slave-5k54"
server_key: "409c0595a34c67ea7d2fb9e15cdb14f4"
server_key_iv: "6188393bcf452a7093a6feb5d8d9256a"
server_key_version: "encryptionkey@0"
I20260812 14:18:06.860713 13695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:06.884838 13695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:06.887238 13695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:06.887405 13695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:06.887693 13695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:06.888020 13695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:06.888075 13695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:06.888118 13695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:06.888144 13695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:06.894150 13695 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:34511
I20260812 14:18:06.894186 13839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:34511 every 8 connection(s)
I20260812 14:18:06.894497 13695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:18:06 dist-test-slave-5k54 krb5kdc[13251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544286, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:06.903542 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:06.895223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:43413 (local address 127.8.85.62:33061)
0812 14:18:06.895291 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:06.895293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:06.895512 (+ 219us) server_negotiation.cc:407] Connection header received
0812 14:18:06.896047 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:06.896049 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:06.896060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:06.896131 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:06.897025 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.897535 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.898531 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.898733 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.898789 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:06.900900 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:06.900902 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:06.900902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:06.900908 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:06.902225 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:06.902628 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:06.902629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:06.902630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:06.902688 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:06.902915 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:06.902915 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:06.902916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:06.903032 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:06.903089 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:06.903343 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 14:18:06.903413 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:18:06.903713 13842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:06.895210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33061 (local address 127.8.85.3:43413)
0812 14:18:06.895361 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:06.895422 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:06.895915 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:06.896186 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:06.896189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:06.896431 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:06.896923 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:06.896929 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.897620 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.897620 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:06.898436 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:06.898438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.898810 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.898811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:06.898909 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:06.899432 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:06.899475 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:06.900807 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:06.902299 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:06.902305 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:06.902307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:06.902529 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:06.902753 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:06.902753 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:06.902754 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:06.902846 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:06.903091 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:06.903095 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:06.903203 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 14:18:06.903303 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 14:18:06.903494 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260812 14:18:06.904259 13840 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33061
I20260812 14:18:06.904367 13840 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:06.904506 13840 heartbeater.cc:511] Master 127.8.85.62:33061 requested a full tablet report, sending...
I20260812 14:18:06.904675 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 13695
I20260812 14:18:06.904779 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:18:06.904949 13302 ts_manager.cc:194] Registered new tserver with Master: 731d5c31effa44049df1aa8623159a2c (127.8.85.3:34511)
I20260812 14:18:06.904976 8532 external_mini_cluster.cc:1428] Setting key 6ab62fbf89664dc0570593cb76f13ede
I20260812 14:18:06.905335 13302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:43413
I20260812 14:18:06.905921 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:18:06.912827 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:06.907222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46218 (local address 127.8.85.62:33061)
0812 14:18:06.907292 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:06.907294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:06.907339 (+ 45us) server_negotiation.cc:407] Connection header received
0812 14:18:06.907399 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:06.907400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:06.907408 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:06.907504 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:06.908051 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.908513 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.909201 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.909369 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.909416 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:06.910482 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:06.910483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:06.910483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:06.910489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:06.911781 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:06.912091 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:06.912092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:06.912092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:06.912161 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:06.912338 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:06.912338 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:06.912339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:06.912528 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:06.912606 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:06.912711 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 14:18:06.912769 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:18:06.914518 13302 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:18:06.916188 13302 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:18:06.922472 13856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:06.918436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:40103 (local address 127.0.0.1:49032)
0812 14:18:06.918575 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:06.918609 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:06.918660 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:06.918933 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:06.918936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:06.918938 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:06.919012 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:06.919014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.920148 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.920149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:06.921329 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:06.921332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.922155 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.922156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:06.922243 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:06.922259 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 14:18:06.922333 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 14:18:06.922387 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
I20260812 14:18:06.922500 13857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:06.918539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49032 (local address 127.8.85.2:40103)
0812 14:18:06.918782 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:06.918785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:06.918793 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:18:06.918827 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:06.918829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:06.918841 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:06.918928 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:06.919075 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.920047 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.921424 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.922063 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.922142 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:06.922198 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:06.922358 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 14:18:06.922398 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":86,"threads_started":1}
I20260812 14:18:06.922771 13860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:06.918762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41120 (local address 127.8.85.1:33717)
0812 14:18:06.918928 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:06.918933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:06.918945 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:18:06.918987 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:06.918989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:06.919006 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:06.919123 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:06.919236 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.920372 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.921585 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.922235 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.922520 (+ 285us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:06.922594 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:06.922624 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 14:18:06.922672 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":105,"threads_started":1}
I20260812 14:18:06.923058 13855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:06.918378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:33717 (local address 127.0.0.1:41120)
0812 14:18:06.918552 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:06.918590 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:06.918653 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:06.919133 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:06.919134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:06.919136 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:06.919173 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:06.919174 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.920474 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.920475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:06.921504 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:06.921506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.922336 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.922337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:06.922411 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:06.922415 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:06.922446 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 14:18:06.922476 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":74,"threads_started":1}
I20260812 14:18:06.923386 13609 tablet_service.cc:1467] Processing CreateTablet for tablet 7ee0f114e21d4a12b4512701879f2d88 (DEFAULT_TABLE table=test-table [id=bef320fa08934d5295ff71ff45a28202]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:06.923794 13609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ee0f114e21d4a12b4512701879f2d88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:06.923820 13452 tablet_service.cc:1467] Processing CreateTablet for tablet 7ee0f114e21d4a12b4512701879f2d88 (DEFAULT_TABLE table=test-table [id=bef320fa08934d5295ff71ff45a28202]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:06.924199 13452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ee0f114e21d4a12b4512701879f2d88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:06.926492 13862 tablet_bootstrap.cc:492] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2: Bootstrap starting.
I20260812 14:18:06.926705 13861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:06.919019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58800 (local address 127.8.85.3:34511)
0812 14:18:06.919400 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:06.919404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:06.919412 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:18:06.919514 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:06.919516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:06.919531 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:06.919602 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:06.919782 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.920704 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.921881 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.922595 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.922664 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:06.922720 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:06.926586 (+ 3866us) server_negotiation.cc:299] Negotiation successful
0812 14:18:06.926635 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":91,"threads_started":1}
I20260812 14:18:06.926910 13858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:06.918598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:34511 (local address 127.0.0.1:58800)
0812 14:18:06.919229 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:06.919239 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:06.919286 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:06.919670 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:06.919671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:06.919673 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:06.919714 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:06.919716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.920793 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.920794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:06.921786 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:06.921789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:06.922771 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:06.922772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:06.922839 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:06.922843 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:06.922895 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 14:18:06.926842 (+ 3947us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":76,"threads_started":1}
I20260812 14:18:06.927139 13864 tablet_bootstrap.cc:492] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4: Bootstrap starting.
I20260812 14:18:06.927309 13862 tablet_bootstrap.cc:654] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:06.927629 13767 tablet_service.cc:1467] Processing CreateTablet for tablet 7ee0f114e21d4a12b4512701879f2d88 (DEFAULT_TABLE table=test-table [id=bef320fa08934d5295ff71ff45a28202]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:06.927912 13864 tablet_bootstrap.cc:654] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:06.927943 13767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ee0f114e21d4a12b4512701879f2d88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:06.928077 13862 log.cc:824] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:06.928632 13864 log.cc:824] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:06.929033 13862 tablet_bootstrap.cc:492] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2: No bootstrap required, opened a new log
I20260812 14:18:06.929121 13862 ts_tablet_manager.cc:1397] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:18:06.929430 13864 tablet_bootstrap.cc:492] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4: No bootstrap required, opened a new log
I20260812 14:18:06.929502 13864 ts_tablet_manager.cc:1397] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:18:06.930879 13867 tablet_bootstrap.cc:492] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c: Bootstrap starting.
I20260812 14:18:06.931524 13867 tablet_bootstrap.cc:654] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:06.931567 13864 raft_consensus.cc:348] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d45f95cece14033a51dbccf68fbc5b4" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33717 } } peers { permanent_uuid: "731d5c31effa44049df1aa8623159a2c" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34511 } } peers { permanent_uuid: "54c66c3be05a4522a9603da6455755d2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 40103 } }
I20260812 14:18:06.931519 13862 raft_consensus.cc:348] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d45f95cece14033a51dbccf68fbc5b4" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33717 } } peers { permanent_uuid: "731d5c31effa44049df1aa8623159a2c" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34511 } } peers { permanent_uuid: "54c66c3be05a4522a9603da6455755d2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 40103 } }
I20260812 14:18:06.931706 13862 raft_consensus.cc:374] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:06.931706 13864 raft_consensus.cc:374] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:06.931743 13864 raft_consensus.cc:729] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d45f95cece14033a51dbccf68fbc5b4, State: Initialized, Role: FOLLOWER
I20260812 14:18:06.931743 13862 raft_consensus.cc:729] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54c66c3be05a4522a9603da6455755d2, State: Initialized, Role: FOLLOWER
I20260812 14:18:06.931887 13862 consensus_queue.cc:260] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d45f95cece14033a51dbccf68fbc5b4" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33717 } } peers { permanent_uuid: "731d5c31effa44049df1aa8623159a2c" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34511 } } peers { permanent_uuid: "54c66c3be05a4522a9603da6455755d2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 40103 } }
I20260812 14:18:06.931963 13864 consensus_queue.cc:260] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d45f95cece14033a51dbccf68fbc5b4" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33717 } } peers { permanent_uuid: "731d5c31effa44049df1aa8623159a2c" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34511 } } peers { permanent_uuid: "54c66c3be05a4522a9603da6455755d2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 40103 } }
I20260812 14:18:06.932188 13867 log.cc:824] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:06.932204 13862 ts_tablet_manager.cc:1428] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:18:06.932248 13864 ts_tablet_manager.cc:1428] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:18:06.932327 13688 heartbeater.cc:503] Master 127.8.85.62:33061 was elected leader, sending a full tablet report...
I20260812 14:18:06.932293 13533 heartbeater.cc:503] Master 127.8.85.62:33061 was elected leader, sending a full tablet report...
I20260812 14:18:06.933317 13867 tablet_bootstrap.cc:492] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c: No bootstrap required, opened a new log
I20260812 14:18:06.933401 13867 ts_tablet_manager.cc:1397] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:18:06.934924 13867 raft_consensus.cc:348] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d45f95cece14033a51dbccf68fbc5b4" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33717 } } peers { permanent_uuid: "731d5c31effa44049df1aa8623159a2c" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34511 } } peers { permanent_uuid: "54c66c3be05a4522a9603da6455755d2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 40103 } }
I20260812 14:18:06.935046 13867 raft_consensus.cc:374] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:06.935068 13867 raft_consensus.cc:729] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 731d5c31effa44049df1aa8623159a2c, State: Initialized, Role: FOLLOWER
I20260812 14:18:06.935141 13867 consensus_queue.cc:260] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d45f95cece14033a51dbccf68fbc5b4" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33717 } } peers { permanent_uuid: "731d5c31effa44049df1aa8623159a2c" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34511 } } peers { permanent_uuid: "54c66c3be05a4522a9603da6455755d2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 40103 } }
I20260812 14:18:06.935375 13867 ts_tablet_manager.cc:1428] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:18:06.935405 13840 heartbeater.cc:503] Master 127.8.85.62:33061 was elected leader, sending a full tablet report...
W20260812 14:18:06.975680 13689 tablet.cc:2369] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:18:07.076030 13534 tablet.cc:2369] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:18:07.145056 13841 tablet.cc:2369] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:18:07.232965 13869 raft_consensus.cc:482] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:18:07.233101 13869 raft_consensus.cc:504] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d45f95cece14033a51dbccf68fbc5b4" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33717 } } peers { permanent_uuid: "731d5c31effa44049df1aa8623159a2c" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34511 } } peers { permanent_uuid: "54c66c3be05a4522a9603da6455755d2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 40103 } }
I20260812 14:18:07.233549 13869 leader_election.cc:288] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 731d5c31effa44049df1aa8623159a2c (127.8.85.3:34511), 54c66c3be05a4522a9603da6455755d2 (127.8.85.2:40103)
I20260812 14:18:07.236608 13857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.233842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:51561 (local address 127.8.85.2:40103)
0812 14:18:07.233888 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:07.233891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:07.233948 (+ 57us) server_negotiation.cc:407] Connection header received
0812 14:18:07.234037 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:07.234038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:07.234054 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:07.234120 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:07.234279 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:07.234756 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:07.235887 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:07.236237 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:07.236325 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:07.236336 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:07.236496 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 14:18:07.236531 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:18:07.236617 13861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.233848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:48655 (local address 127.8.85.3:34511)
0812 14:18:07.233914 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:07.233917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:07.234029 (+ 112us) server_negotiation.cc:407] Connection header received
0812 14:18:07.234083 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:07.234083 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:07.234101 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:07.234170 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:07.234340 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:07.234789 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:07.235875 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:07.236244 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:07.236338 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:07.236348 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:07.236508 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 14:18:07.236549 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:18:07.236644 13872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.233737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:40103 (local address 127.8.85.1:51561)
0812 14:18:07.233882 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:07.233894 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:07.233961 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:07.234125 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:07.234126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:07.234128 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:07.234201 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:07.234205 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:07.234868 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:07.234869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:07.235799 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:07.235802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:07.236326 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:07.236327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:07.236403 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:07.236407 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:07.236488 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 14:18:07.236539 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":63,"thread_start_us":66,"threads_started":1}
I20260812 14:18:07.236646 13873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.233741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:34511 (local address 127.8.85.1:48655)
0812 14:18:07.233941 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:07.233951 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:07.234018 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:07.234175 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:07.234175 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:07.234178 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:07.234254 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:07.234256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:07.234887 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:07.234888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:07.235776 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:07.235778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:07.236338 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:07.236338 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:07.236414 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:07.236418 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:07.236490 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:18:07.236537 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":12,"thread_start_us":137,"threads_started":1}
I20260812 14:18:07.237068 13788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee0f114e21d4a12b4512701879f2d88" candidate_uuid: "5d45f95cece14033a51dbccf68fbc5b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "731d5c31effa44049df1aa8623159a2c" is_pre_election: true
I20260812 14:18:07.237139 13634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee0f114e21d4a12b4512701879f2d88" candidate_uuid: "5d45f95cece14033a51dbccf68fbc5b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54c66c3be05a4522a9603da6455755d2" is_pre_election: true
I20260812 14:18:07.237221 13788 raft_consensus.cc:2454] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d45f95cece14033a51dbccf68fbc5b4 in term 0.
I20260812 14:18:07.237308 13634 raft_consensus.cc:2454] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d45f95cece14033a51dbccf68fbc5b4 in term 0.
I20260812 14:18:07.237457 13395 leader_election.cc:302] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d45f95cece14033a51dbccf68fbc5b4, 731d5c31effa44049df1aa8623159a2c; no voters:
I20260812 14:18:07.237603 13869 raft_consensus.cc:2781] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:18:07.237663 13869 raft_consensus.cc:482] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:18:07.237699 13869 raft_consensus.cc:3037] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:07.238391 13869 raft_consensus.cc:504] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d45f95cece14033a51dbccf68fbc5b4" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33717 } } peers { permanent_uuid: "731d5c31effa44049df1aa8623159a2c" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34511 } } peers { permanent_uuid: "54c66c3be05a4522a9603da6455755d2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 40103 } }
I20260812 14:18:07.238540 13869 leader_election.cc:288] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [CANDIDATE]: Term 1 election: Requested vote from peers 731d5c31effa44049df1aa8623159a2c (127.8.85.3:34511), 54c66c3be05a4522a9603da6455755d2 (127.8.85.2:40103)
I20260812 14:18:07.238698 13634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee0f114e21d4a12b4512701879f2d88" candidate_uuid: "5d45f95cece14033a51dbccf68fbc5b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54c66c3be05a4522a9603da6455755d2"
I20260812 14:18:07.238699 13788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee0f114e21d4a12b4512701879f2d88" candidate_uuid: "5d45f95cece14033a51dbccf68fbc5b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "731d5c31effa44049df1aa8623159a2c"
I20260812 14:18:07.238776 13634 raft_consensus.cc:3037] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:07.238786 13788 raft_consensus.cc:3037] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:07.239769 13634 raft_consensus.cc:2454] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d45f95cece14033a51dbccf68fbc5b4 in term 1.
I20260812 14:18:07.239778 13788 raft_consensus.cc:2454] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d45f95cece14033a51dbccf68fbc5b4 in term 1.
I20260812 14:18:07.239956 13396 leader_election.cc:302] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54c66c3be05a4522a9603da6455755d2, 5d45f95cece14033a51dbccf68fbc5b4; no voters:
I20260812 14:18:07.240079 13869 raft_consensus.cc:2781] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:07.240234 13869 raft_consensus.cc:686] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [term 1 LEADER]: Becoming Leader. State: Replica: 5d45f95cece14033a51dbccf68fbc5b4, State: Running, Role: LEADER
I20260812 14:18:07.240350 13869 consensus_queue.cc:237] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d45f95cece14033a51dbccf68fbc5b4" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33717 } } peers { permanent_uuid: "731d5c31effa44049df1aa8623159a2c" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34511 } } peers { permanent_uuid: "54c66c3be05a4522a9603da6455755d2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 40103 } }
I20260812 14:18:07.241217 13302 catalog_manager.cc:5495] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d45f95cece14033a51dbccf68fbc5b4 (127.8.85.1). New cstate: current_term: 1 leader_uuid: "5d45f95cece14033a51dbccf68fbc5b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d45f95cece14033a51dbccf68fbc5b4" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "731d5c31effa44049df1aa8623159a2c" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54c66c3be05a4522a9603da6455755d2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 40103 } health_report { overall_health: UNKNOWN } } }
I20260812 14:18:07.273965 13860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.271729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41122 (local address 127.8.85.1:33717)
0812 14:18:07.271795 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:07.271796 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:07.271805 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:18:07.271841 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:07.271842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:07.271865 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:07.271940 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:18:07.272086 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:07.272543 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:07.273310 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:07.273500 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:07.273555 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:07.273699 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:18:07.273796 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:18:07.273862 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:07.273890 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 14:18:07.273907 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:18:07.275929 13788 raft_consensus.cc:1261] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c [term 1 FOLLOWER]: Refusing update from remote peer 5d45f95cece14033a51dbccf68fbc5b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:07.275928 13634 raft_consensus.cc:1261] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2 [term 1 FOLLOWER]: Refusing update from remote peer 5d45f95cece14033a51dbccf68fbc5b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:07.276326 13874 consensus_queue.cc:1035] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54c66c3be05a4522a9603da6455755d2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 40103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:07.276484 13869 consensus_queue.cc:1035] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "731d5c31effa44049df1aa8623159a2c" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:07.279392 13878 mvcc.cc:205] Tried to move back new op lower bound from 7317685400678936576 to 7317685400537055232. Current Snapshot: MvccSnapshot[applied={T|T < 7317685400678936576}]
I20260812 14:18:07.279541 13879 mvcc.cc:205] Tried to move back new op lower bound from 7317685400678936576 to 7317685400537055232. Current Snapshot: MvccSnapshot[applied={T|T < 7317685400678936576}]
I20260812 14:18:07.281044 13302 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:07.281152 13302 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:07.281950 13302 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0261eaf5a528441da33bf6883f9da222: Sending DeleteTablet for 3 replicas of tablet 7ee0f114e21d4a12b4512701879f2d88
I20260812 14:18:07.282258 13609 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ee0f114e21d4a12b4512701879f2d88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:07 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:49032
I20260812 14:18:07.282343 13452 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ee0f114e21d4a12b4512701879f2d88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:07 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:41120
I20260812 14:18:07.282492 13887 tablet_replica.cc:330] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2: stopping tablet replica
I20260812 14:18:07.282495 13767 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ee0f114e21d4a12b4512701879f2d88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:07 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:58800
I20260812 14:18:07.282552 13888 tablet_replica.cc:330] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4: stopping tablet replica
I20260812 14:18:07.282589 13887 raft_consensus.cc:2229] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:07.282621 13888 raft_consensus.cc:2229] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:18:07.282696 13887 raft_consensus.cc:2258] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:07.282701 13888 raft_consensus.cc:2258] T 7ee0f114e21d4a12b4512701879f2d88 P 5d45f95cece14033a51dbccf68fbc5b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:07.283110 13887 ts_tablet_manager.cc:1905] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:07.283144 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 13378
I20260812 14:18:07.285349 13887 ts_tablet_manager.cc:1918] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:07.285410 13887 log.cc:1196] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/7ee0f114e21d4a12b4512701879f2d88
I20260812 14:18:07.285598 13887 ts_tablet_manager.cc:1939] T 7ee0f114e21d4a12b4512701879f2d88 P 54c66c3be05a4522a9603da6455755d2: Deleting consensus metadata
I20260812 14:18:07.286238 13284 catalog_manager.cc:4841] TS 54c66c3be05a4522a9603da6455755d2 (127.8.85.2:40103): tablet 7ee0f114e21d4a12b4512701879f2d88 (table test-table [id=bef320fa08934d5295ff71ff45a28202]) successfully deleted
I20260812 14:18:07.287829 13889 tablet_replica.cc:330] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c: stopping tablet replica
I20260812 14:18:07.287923 13889 raft_consensus.cc:2229] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:07.288045 13889 raft_consensus.cc:2258] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:07.288516 13889 ts_tablet_manager.cc:1905] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:07.289848 13889 ts_tablet_manager.cc:1918] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:07.289937 13889 log.cc:1196] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/7ee0f114e21d4a12b4512701879f2d88
I20260812 14:18:07.290182 13889 ts_tablet_manager.cc:1939] T 7ee0f114e21d4a12b4512701879f2d88 P 731d5c31effa44049df1aa8623159a2c: Deleting consensus metadata
I20260812 14:18:07.290577 13283 catalog_manager.cc:4841] TS 731d5c31effa44049df1aa8623159a2c (127.8.85.3:34511): tablet 7ee0f114e21d4a12b4512701879f2d88 (table test-table [id=bef320fa08934d5295ff71ff45a28202]) successfully deleted
W20260812 14:18:07.292961 13283 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:33717 (error 108)
I20260812 14:18:07.293154 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 13539
I20260812 14:18:07.294241 13858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.294115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:33717 (local address 127.0.0.1:41138)
0812 14:18:07.294175 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:07.294213 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:33717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 14:18:07.294381 13283 catalog_manager.cc:4584] TS 5d45f95cece14033a51dbccf68fbc5b4 (127.8.85.1:33717): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ee0f114e21d4a12b4512701879f2d88: Network error: Client connection negotiation failed: client connection to 127.8.85.1:33717: connect: Connection refused (error 111)
I20260812 14:18:07.300139 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 13695
I20260812 14:18:07.307174 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 13264
2026-08-12T14:18:07Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2322 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T14:18:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:07Z Disabled control of system clock
I20260812 14:18:07.338133 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:38723
--webserver_interface=127.8.85.62
--webserver_port=0
--builtin_ntp_servers=127.8.85.20:44221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:38723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 14:18:07.419775 13895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:07.419978 13895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:07.419999 13895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:07.421154 13895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:07.421197 13895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:07.421211 13895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:07.421223 13895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:07.421275 13895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:07.422705 13895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:44221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:38723
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:38723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:07.422953 13895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:07.423255 13895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:07.428108 13903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:07.428261 13895 server_base.cc:1034] running on GCE node
W20260812 14:18:07.428149 13907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:07.428112 13904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:07.428599 13895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:07.428920 13895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:07.430089 13895 hybrid_clock.cc:648] HybridClock initialized: now 1786544287430082 us; error 38 us; skew 500 ppm
I20260812 14:18:07.431389 13895 webserver.cc:460] Webserver started at http://127.8.85.62:45003/ using document root <none> and password file <none>
I20260812 14:18:07.431665 13895 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:07.431739 13895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:07.431902 13895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:07.432981 13895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "7ed08ee1ca1640a2a4ae3dcde8587332"
format_stamp: "Formatted at 2026-08-12 14:18:07 on dist-test-slave-5k54"
server_key: "88906e93a8e5287134aad91c8e575c13"
server_key_iv: "7be96e64e20715623d8cf1cff272443e"
server_key_version: "encryptionkey@0"
I20260812 14:18:07.433436 13895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "7ed08ee1ca1640a2a4ae3dcde8587332"
format_stamp: "Formatted at 2026-08-12 14:18:07 on dist-test-slave-5k54"
server_key: "88906e93a8e5287134aad91c8e575c13"
server_key_iv: "7be96e64e20715623d8cf1cff272443e"
server_key_version: "encryptionkey@0"
I20260812 14:18:07.434816 13895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:18:07.435600 13916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:07.435948 13895 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:07.436084 13895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "7ed08ee1ca1640a2a4ae3dcde8587332"
format_stamp: "Formatted at 2026-08-12 14:18:07 on dist-test-slave-5k54"
server_key: "88906e93a8e5287134aad91c8e575c13"
server_key_iv: "7be96e64e20715623d8cf1cff272443e"
server_key_version: "encryptionkey@0"
I20260812 14:18:07.436177 13895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:07.439575 13895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:07.439853 13895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:07.440001 13895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:07.444149 13895 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:38723
I20260812 14:18:07.444178 13980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:38723 every 8 connection(s)
I20260812 14:18:07.444497 13895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:07.445152 13981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:07.448262 13981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332: Bootstrap starting.
I20260812 14:18:07.448873 13981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:07.449124 13981 log.cc:824] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:07.449740 13981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332: No bootstrap required, opened a new log
I20260812 14:18:07.451233 13981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed08ee1ca1640a2a4ae3dcde8587332" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38723 } }
I20260812 14:18:07.451367 13981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:07.451390 13981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ed08ee1ca1640a2a4ae3dcde8587332, State: Initialized, Role: FOLLOWER
I20260812 14:18:07.451594 13981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed08ee1ca1640a2a4ae3dcde8587332" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38723 } }
I20260812 14:18:07.451683 13981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:07.451706 13981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:07.451733 13981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:07.452204 13981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed08ee1ca1640a2a4ae3dcde8587332" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38723 } }
I20260812 14:18:07.452330 13981 leader_election.cc:302] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ed08ee1ca1640a2a4ae3dcde8587332; no voters:
I20260812 14:18:07.452447 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 13895
I20260812 14:18:07.452513 13981 leader_election.cc:288] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:07.452566 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:07.452594 13984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:07.452752 8532 external_mini_cluster.cc:1428] Setting key a2ba44b982cf025b1e80f336a47d7639
I20260812 14:18:07.452772 13984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [term 1 LEADER]: Becoming Leader. State: Replica: 7ed08ee1ca1640a2a4ae3dcde8587332, State: Running, Role: LEADER
I20260812 14:18:07.452829 13981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:07.452849 13984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed08ee1ca1640a2a4ae3dcde8587332" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38723 } }
I20260812 14:18:07.453359 13985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ed08ee1ca1640a2a4ae3dcde8587332" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed08ee1ca1640a2a4ae3dcde8587332" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38723 } } }
I20260812 14:18:07.453403 13986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ed08ee1ca1640a2a4ae3dcde8587332. Latest consensus state: current_term: 1 leader_uuid: "7ed08ee1ca1640a2a4ae3dcde8587332" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed08ee1ca1640a2a4ae3dcde8587332" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38723 } } }
I20260812 14:18:07.453863 13986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:07.454071 13985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332 [sys.catalog]: This master's current role is: LEADER
W20260812 14:18:07.454825 14009 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:18:07.454929 14009 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:18:07.455009 14003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:07.455281 14003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:07.455472 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.453657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44544 (local address 127.8.85.62:38723)
0812 14:18:07.453872 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:07.453880 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:07.453914 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:18:07.454037 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:07.454045 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:07.454091 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:07.454236 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:07.454966 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:07.454969 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:07.454970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:07.454973 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:07.455018 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:07.455033 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:07.455102 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:07.455190 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 14:18:07.455250 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":100,"threads_started":1}
I20260812 14:18:07.457034 14003 catalog_manager.cc:1347] Generated new cluster ID: 2376b322943f4e3ea9eca5c70a87e9a9
I20260812 14:18:07.457079 14003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:18:07.473606 14003 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:07.473804 14003 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:07.480834 14003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332: Generated new TSK 0
I20260812 14:18:07.481042 14003 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:18:07.489328 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38723
--builtin_ntp_servers=127.8.85.20:44221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 14:18:07.572697 14013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:07.572897 14013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:07.572921 14013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:07.574054 14013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:07.574091 14013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:07.574115 14013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:18:07.575567 14013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:44221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:07.575805 14013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:07.576110 14013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:07.580986 14019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:07.580981 14020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:07.580981 14022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:07.581214 14013 server_base.cc:1034] running on GCE node
I20260812 14:18:07.581435 14013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:07.581637 14013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:07.582777 14013 hybrid_clock.cc:648] HybridClock initialized: now 1786544287582760 us; error 31 us; skew 500 ppm
I20260812 14:18:07.583984 14013 webserver.cc:460] Webserver started at http://127.8.85.1:39029/ using document root <none> and password file <none>
I20260812 14:18:07.584179 14013 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:07.584219 14013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:07.584309 14013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:07.585425 14013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "d33f3ef709c04145b05e6bf84a947af9"
format_stamp: "Formatted at 2026-08-12 14:18:07 on dist-test-slave-5k54"
server_key: "b8dab510bee145dc5f181423575a7daf"
server_key_iv: "322000cbf30784fa3c0374be8c4f5805"
server_key_version: "encryptionkey@0"
I20260812 14:18:07.585729 14013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "d33f3ef709c04145b05e6bf84a947af9"
format_stamp: "Formatted at 2026-08-12 14:18:07 on dist-test-slave-5k54"
server_key: "b8dab510bee145dc5f181423575a7daf"
server_key_iv: "322000cbf30784fa3c0374be8c4f5805"
server_key_version: "encryptionkey@0"
I20260812 14:18:07.586922 14013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:18:07.587605 14028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:07.587764 14013 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:07.587850 14013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "d33f3ef709c04145b05e6bf84a947af9"
format_stamp: "Formatted at 2026-08-12 14:18:07 on dist-test-slave-5k54"
server_key: "b8dab510bee145dc5f181423575a7daf"
server_key_iv: "322000cbf30784fa3c0374be8c4f5805"
server_key_version: "encryptionkey@0"
I20260812 14:18:07.587905 14013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:07.591110 14013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:07.591316 14013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:07.591420 14013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:07.591652 14013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:07.591897 14013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:07.591933 14013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:07.591959 14013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:07.591971 14013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:07.597352 14013 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:41779
I20260812 14:18:07.597436 14162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:41779 every 8 connection(s)
I20260812 14:18:07.597757 14013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:18:07.600711 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.598482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:56671 (local address 127.8.85.62:38723)
0812 14:18:07.598553 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:07.598555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:07.598789 (+ 234us) server_negotiation.cc:407] Connection header received
0812 14:18:07.599237 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:07.599238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:07.599243 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:07.599300 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:07.600510 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:07.600511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:07.600511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:07.600514 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:07.600526 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:07.600530 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:07.600574 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:07.600648 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 14:18:07.600660 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:18:07.600898 14166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.598467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38723 (local address 127.8.85.1:56671)
0812 14:18:07.598671 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:07.598705 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:07.599146 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:07.599376 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:07.599380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:07.599388 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:07.600378 (+ 990us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:18:07.600378 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:07.600406 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:18:07.600433 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:07.600587 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:07.600590 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:07.600592 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:18:07.600644 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 14:18:07.600691 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":101,"threads_started":1}
I20260812 14:18:07.601370 14164 heartbeater.cc:347] Connected to a master server at 127.8.85.62:38723
I20260812 14:18:07.601451 14164 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:07.601613 14164 heartbeater.cc:511] Master 127.8.85.62:38723 requested a full tablet report, sending...
I20260812 14:18:07.602092 13934 ts_manager.cc:194] Registered new tserver with Master: d33f3ef709c04145b05e6bf84a947af9 (127.8.85.1:41779)
I20260812 14:18:07.604997 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 14013
I20260812 14:18:07.605108 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:18:07.605298 8532 external_mini_cluster.cc:1428] Setting key 92f09f3a94cb6ff675323e097d705785
I20260812 14:18:07.606153 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38723
--builtin_ntp_servers=127.8.85.20:44221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 14:18:07.689141 14167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:07.689324 14167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:07.689345 14167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:07.690647 14167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:07.690690 14167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:07.690721 14167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:18:07.692391 14167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:44221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:07.692708 14167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:07.693068 14167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:07.698356 14177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:07.698351 14180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:07.698591 14167 server_base.cc:1034] running on GCE node
W20260812 14:18:07.698407 14176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:07.698868 14167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:07.699107 14167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:07.700248 14167 hybrid_clock.cc:648] HybridClock initialized: now 1786544287700230 us; error 36 us; skew 500 ppm
I20260812 14:18:07.701452 14167 webserver.cc:460] Webserver started at http://127.8.85.2:34601/ using document root <none> and password file <none>
I20260812 14:18:07.701701 14167 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:07.701747 14167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:07.701936 14167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:07.702973 14167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "38ddf18e227c464db09058d168ecd02a"
format_stamp: "Formatted at 2026-08-12 14:18:07 on dist-test-slave-5k54"
server_key: "8f9e3b445d6aabc14e69bf701f5ac6a5"
server_key_iv: "970433ae100519b28c147eba7a367ac8"
server_key_version: "encryptionkey@0"
I20260812 14:18:07.703341 14167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "38ddf18e227c464db09058d168ecd02a"
format_stamp: "Formatted at 2026-08-12 14:18:07 on dist-test-slave-5k54"
server_key: "8f9e3b445d6aabc14e69bf701f5ac6a5"
server_key_iv: "970433ae100519b28c147eba7a367ac8"
server_key_version: "encryptionkey@0"
I20260812 14:18:07.704784 14167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:18:07.705498 14186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:07.705665 14167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:07.705770 14167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "38ddf18e227c464db09058d168ecd02a"
format_stamp: "Formatted at 2026-08-12 14:18:07 on dist-test-slave-5k54"
server_key: "8f9e3b445d6aabc14e69bf701f5ac6a5"
server_key_iv: "970433ae100519b28c147eba7a367ac8"
server_key_version: "encryptionkey@0"
I20260812 14:18:07.705842 14167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:07.709504 14167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:07.709762 14167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:07.709918 14167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:07.710130 14167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:07.710439 14167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:07.710495 14167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:07.710536 14167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:07.710563 14167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:07.716171 14167 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:36977
I20260812 14:18:07.716216 14321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:36977 every 8 connection(s)
I20260812 14:18:07.716521 14167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:18:07.719331 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.717181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:34795 (local address 127.8.85.62:38723)
0812 14:18:07.717224 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:07.717226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:07.717451 (+ 225us) server_negotiation.cc:407] Connection header received
0812 14:18:07.717935 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:07.717936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:07.717943 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:07.717992 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:07.719086 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:07.719088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:07.719088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:07.719093 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:07.719113 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:07.719117 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:07.719173 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:07.719244 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 14:18:07.719260 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:18:07.719563 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.717148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38723 (local address 127.8.85.2:34795)
0812 14:18:07.717321 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:07.717358 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:07.717824 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:07.718076 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:07.718080 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:07.718090 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:07.718934 (+ 844us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:18:07.718934 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:07.718962 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:18:07.718989 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:07.719175 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:07.719180 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:07.719183 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:18:07.719254 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 14:18:07.719314 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260812 14:18:07.720021 14322 heartbeater.cc:347] Connected to a master server at 127.8.85.62:38723
I20260812 14:18:07.720095 14322 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:07.720261 14322 heartbeater.cc:511] Master 127.8.85.62:38723 requested a full tablet report, sending...
I20260812 14:18:07.720630 13934 ts_manager.cc:194] Registered new tserver with Master: 38ddf18e227c464db09058d168ecd02a (127.8.85.2:36977)
I20260812 14:18:07.721585 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 14167
I20260812 14:18:07.721668 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:18:07.721803 8532 external_mini_cluster.cc:1428] Setting key a5b4116e774081eb6443955a3570ec8f
I20260812 14:18:07.722643 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38723
--builtin_ntp_servers=127.8.85.20:44221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 14:18:07.806155 14326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:07.806356 14326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:07.806378 14326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:07.807544 14326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:07.807585 14326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:07.807611 14326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:18:07.809129 14326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:44221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:07.809324 14326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:07.809592 14326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:07.814824 14331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:07.814970 14332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:07.814854 14326 server_base.cc:1034] running on GCE node
W20260812 14:18:07.814833 14334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:07.815286 14326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:07.815623 14326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:07.816792 14326 hybrid_clock.cc:648] HybridClock initialized: now 1786544287816782 us; error 39 us; skew 500 ppm
I20260812 14:18:07.818061 14326 webserver.cc:460] Webserver started at http://127.8.85.3:33533/ using document root <none> and password file <none>
I20260812 14:18:07.818317 14326 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:07.818388 14326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:07.818547 14326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:07.819723 14326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "df9781e00a36431b928f661b7fcb5f27"
format_stamp: "Formatted at 2026-08-12 14:18:07 on dist-test-slave-5k54"
server_key: "025670626b66619a2a4df736eb323626"
server_key_iv: "37adff8fd553c46451717cbb6021ff6c"
server_key_version: "encryptionkey@0"
I20260812 14:18:07.820158 14326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "df9781e00a36431b928f661b7fcb5f27"
format_stamp: "Formatted at 2026-08-12 14:18:07 on dist-test-slave-5k54"
server_key: "025670626b66619a2a4df736eb323626"
server_key_iv: "37adff8fd553c46451717cbb6021ff6c"
server_key_version: "encryptionkey@0"
I20260812 14:18:07.821591 14326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:07.822388 14341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:07.822577 14326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:07.822685 14326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "df9781e00a36431b928f661b7fcb5f27"
format_stamp: "Formatted at 2026-08-12 14:18:07 on dist-test-slave-5k54"
server_key: "025670626b66619a2a4df736eb323626"
server_key_iv: "37adff8fd553c46451717cbb6021ff6c"
server_key_version: "encryptionkey@0"
I20260812 14:18:07.822760 14326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:07.826301 14326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:07.826567 14326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:07.826709 14326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:07.826910 14326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:07.827214 14326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:07.827270 14326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:07.827308 14326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:07.827335 14326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:07.833760 14326 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:44913
I20260812 14:18:07.833897 14466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:44913 every 8 connection(s)
I20260812 14:18:07.834177 14326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:18:07.837182 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.834976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:35077 (local address 127.8.85.62:38723)
0812 14:18:07.835021 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:07.835023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:07.835246 (+ 223us) server_negotiation.cc:407] Connection header received
0812 14:18:07.835800 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:07.835801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:07.835808 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:07.835874 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:07.836985 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:07.836987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:07.836987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:07.836992 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:07.837008 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:07.837013 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:07.837066 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:07.837112 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 14:18:07.837128 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:18:07.837390 14470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.834927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38723 (local address 127.8.85.3:35077)
0812 14:18:07.835115 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:07.835155 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:07.835707 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:07.835927 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:07.835930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:07.835940 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:07.836825 (+ 885us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:18:07.836825 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:07.836854 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:18:07.836882 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:07.837063 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:07.837065 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:07.837067 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:18:07.837118 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:18:07.837165 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260812 14:18:07.837888 14468 heartbeater.cc:347] Connected to a master server at 127.8.85.62:38723
I20260812 14:18:07.837972 14468 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:07.838150 14468 heartbeater.cc:511] Master 127.8.85.62:38723 requested a full tablet report, sending...
I20260812 14:18:07.838258 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 14326
I20260812 14:18:07.838361 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:18:07.838564 8532 external_mini_cluster.cc:1428] Setting key 287c5a48414c4bb00067dd1cc1181c0c
I20260812 14:18:07.838582 13934 ts_manager.cc:194] Registered new tserver with Master: df9781e00a36431b928f661b7fcb5f27 (127.8.85.3:44913)
I20260812 14:18:07.839512 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:18:07.841869 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.840879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44556 (local address 127.8.85.62:38723)
0812 14:18:07.840942 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:07.840944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:07.841083 (+ 139us) server_negotiation.cc:407] Connection header received
0812 14:18:07.841120 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:07.841121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:07.841125 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:07.841170 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:07.841670 (+ 500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:07.841671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:07.841671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:07.841674 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:07.841687 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:07.841690 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:07.841733 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:07.841763 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 14:18:07.841773 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:18:07.843573 13934 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:18:07.845079 13934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:18:07.848937 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.847554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37586 (local address 127.8.85.2:36977)
0812 14:18:07.847756 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:07.847760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:07.847770 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:18:07.847796 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:07.847798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:07.847807 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:07.847879 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:07.848613 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:07.848622 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:07.848622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:07.848627 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:07.848661 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:07.848692 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:07.848759 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:07.848844 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 14:18:07.848861 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":92,"threads_started":1}
I20260812 14:18:07.848922 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.847318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:36977 (local address 127.0.0.1:37586)
0812 14:18:07.847537 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:07.847549 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:07.847598 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:07.848186 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:07.848187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:07.848189 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:07.848515 (+ 326us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:18:07.848515 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:07.848521 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:18:07.848530 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:07.848765 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:07.848772 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:07.848775 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:18:07.848839 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 14:18:07.848854 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":84,"threads_started":1}
I20260812 14:18:07.849089 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.847190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:44913 (local address 127.0.0.1:41238)
0812 14:18:07.847540 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:07.847550 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:07.847598 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:07.847906 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:07.847913 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:07.847920 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:07.848397 (+ 477us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:18:07.848398 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:07.848416 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:18:07.848441 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:07.848938 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:07.848939 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:07.848940 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 14:18:07.848988 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 14:18:07.849003 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":88,"threads_started":1}
I20260812 14:18:07.849190 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.847227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57172 (local address 127.8.85.1:41779)
0812 14:18:07.847463 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:07.847468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:07.847479 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:18:07.847560 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:07.847573 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:07.847586 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:07.847646 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:07.848848 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:07.848855 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:07.848855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:07.848861 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:07.848892 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:07.848919 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:07.848958 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:07.849078 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 14:18:07.849093 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":80,"threads_started":1}
I20260812 14:18:07.849349 14482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.847344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.8.85.3:44913)
0812 14:18:07.847711 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:07.847715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:07.847725 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:18:07.847751 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:07.847753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:07.847759 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:07.847845 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:07.848782 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:07.848786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:07.848787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:07.848791 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:07.848824 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:07.848838 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:07.848909 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:07.849240 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 14:18:07.849256 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":106,"threads_started":1}
I20260812 14:18:07.849397 14477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:07.847097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:41779 (local address 127.0.0.1:57172)
0812 14:18:07.847301 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:07.847341 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:07.847397 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:07.848274 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:07.848275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:07.848277 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:07.848717 (+ 440us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:18:07.848718 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:07.848725 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:18:07.848736 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:07.849006 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:07.849006 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:07.849006 (+ 0us) client_negotiation.cc:769] Sending connection context
0812 14:18:07.849308 (+ 302us) client_negotiation.cc:240] Negotiation successful
0812 14:18:07.849321 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":84,"threads_started":1}
I20260812 14:18:07.849968 14397 tablet_service.cc:1467] Processing CreateTablet for tablet c9f93cb5c8d9444fa98e7900f863daa9 (DEFAULT_TABLE table=test-table [id=c640a64119d14a05b0e7a17f581fff4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:07.849969 14245 tablet_service.cc:1467] Processing CreateTablet for tablet c9f93cb5c8d9444fa98e7900f863daa9 (DEFAULT_TABLE table=test-table [id=c640a64119d14a05b0e7a17f581fff4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:07.850325 14397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9f93cb5c8d9444fa98e7900f863daa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:07.850325 14245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9f93cb5c8d9444fa98e7900f863daa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:07.851818 14083 tablet_service.cc:1467] Processing CreateTablet for tablet c9f93cb5c8d9444fa98e7900f863daa9 (DEFAULT_TABLE table=test-table [id=c640a64119d14a05b0e7a17f581fff4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:07.852181 14083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9f93cb5c8d9444fa98e7900f863daa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:07.853859 14487 tablet_bootstrap.cc:492] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27: Bootstrap starting.
I20260812 14:18:07.854514 14487 tablet_bootstrap.cc:654] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:07.854826 14487 log.cc:824] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:07.855080 14488 tablet_bootstrap.cc:492] T c9f93cb5c8d9444fa98e7900f863daa9 P d33f3ef709c04145b05e6bf84a947af9: Bootstrap starting.
I20260812 14:18:07.855170 14486 tablet_bootstrap.cc:492] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a: Bootstrap starting.
I20260812 14:18:07.855855 14488 tablet_bootstrap.cc:654] T c9f93cb5c8d9444fa98e7900f863daa9 P d33f3ef709c04145b05e6bf84a947af9: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:07.855866 14486 tablet_bootstrap.cc:654] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:07.855872 14487 tablet_bootstrap.cc:492] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27: No bootstrap required, opened a new log
I20260812 14:18:07.855960 14487 ts_tablet_manager.cc:1397] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:18:07.856200 14486 log.cc:824] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:07.856200 14488 log.cc:824] T c9f93cb5c8d9444fa98e7900f863daa9 P d33f3ef709c04145b05e6bf84a947af9: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:07.856950 14486 tablet_bootstrap.cc:492] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a: No bootstrap required, opened a new log
I20260812 14:18:07.856950 14488 tablet_bootstrap.cc:492] T c9f93cb5c8d9444fa98e7900f863daa9 P d33f3ef709c04145b05e6bf84a947af9: No bootstrap required, opened a new log
I20260812 14:18:07.857021 14486 ts_tablet_manager.cc:1397] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 14:18:07.857021 14488 ts_tablet_manager.cc:1397] T c9f93cb5c8d9444fa98e7900f863daa9 P d33f3ef709c04145b05e6bf84a947af9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:18:07.858111 14487 raft_consensus.cc:348] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33f3ef709c04145b05e6bf84a947af9" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 41779 } } peers { permanent_uuid: "df9781e00a36431b928f661b7fcb5f27" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44913 } } peers { permanent_uuid: "38ddf18e227c464db09058d168ecd02a" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 36977 } }
I20260812 14:18:07.858235 14487 raft_consensus.cc:374] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:07.858256 14487 raft_consensus.cc:729] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df9781e00a36431b928f661b7fcb5f27, State: Initialized, Role: FOLLOWER
I20260812 14:18:07.858404 14487 consensus_queue.cc:260] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33f3ef709c04145b05e6bf84a947af9" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 41779 } } peers { permanent_uuid: "df9781e00a36431b928f661b7fcb5f27" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44913 } } peers { permanent_uuid: "38ddf18e227c464db09058d168ecd02a" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 36977 } }
I20260812 14:18:07.858683 14487 ts_tablet_manager.cc:1428] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:18:07.858811 14468 heartbeater.cc:503] Master 127.8.85.62:38723 was elected leader, sending a full tablet report...
I20260812 14:18:07.859681 14488 raft_consensus.cc:348] T c9f93cb5c8d9444fa98e7900f863daa9 P d33f3ef709c04145b05e6bf84a947af9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33f3ef709c04145b05e6bf84a947af9" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 41779 } } peers { permanent_uuid: "df9781e00a36431b928f661b7fcb5f27" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44913 } } peers { permanent_uuid: "38ddf18e227c464db09058d168ecd02a" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 36977 } }
I20260812 14:18:07.859681 14486 raft_consensus.cc:348] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33f3ef709c04145b05e6bf84a947af9" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 41779 } } peers { permanent_uuid: "df9781e00a36431b928f661b7fcb5f27" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44913 } } peers { permanent_uuid: "38ddf18e227c464db09058d168ecd02a" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 36977 } }
I20260812 14:18:07.859843 14488 raft_consensus.cc:374] T c9f93cb5c8d9444fa98e7900f863daa9 P d33f3ef709c04145b05e6bf84a947af9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:07.859843 14486 raft_consensus.cc:374] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:07.859884 14488 raft_consensus.cc:729] T c9f93cb5c8d9444fa98e7900f863daa9 P d33f3ef709c04145b05e6bf84a947af9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d33f3ef709c04145b05e6bf84a947af9, State: Initialized, Role: FOLLOWER
I20260812 14:18:07.859892 14486 raft_consensus.cc:729] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38ddf18e227c464db09058d168ecd02a, State: Initialized, Role: FOLLOWER
I20260812 14:18:07.860008 14486 consensus_queue.cc:260] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33f3ef709c04145b05e6bf84a947af9" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 41779 } } peers { permanent_uuid: "df9781e00a36431b928f661b7fcb5f27" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44913 } } peers { permanent_uuid: "38ddf18e227c464db09058d168ecd02a" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 36977 } }
I20260812 14:18:07.860080 14488 consensus_queue.cc:260] T c9f93cb5c8d9444fa98e7900f863daa9 P d33f3ef709c04145b05e6bf84a947af9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33f3ef709c04145b05e6bf84a947af9" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 41779 } } peers { permanent_uuid: "df9781e00a36431b928f661b7fcb5f27" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44913 } } peers { permanent_uuid: "38ddf18e227c464db09058d168ecd02a" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 36977 } }
I20260812 14:18:07.860338 14486 ts_tablet_manager.cc:1428] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 14:18:07.860338 14488 ts_tablet_manager.cc:1428] T c9f93cb5c8d9444fa98e7900f863daa9 P d33f3ef709c04145b05e6bf84a947af9: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 14:18:07.860451 14322 heartbeater.cc:503] Master 127.8.85.62:38723 was elected leader, sending a full tablet report...
I20260812 14:18:07.860502 14164 heartbeater.cc:503] Master 127.8.85.62:38723 was elected leader, sending a full tablet report...
W20260812 14:18:07.967070 14323 tablet.cc:2369] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:18:08.084757 14469 tablet.cc:2369] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:18:08.098448 14165 tablet.cc:2369] T c9f93cb5c8d9444fa98e7900f863daa9 P d33f3ef709c04145b05e6bf84a947af9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:18:08.120491 14494 raft_consensus.cc:482] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:18:08.120625 14494 raft_consensus.cc:504] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33f3ef709c04145b05e6bf84a947af9" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 41779 } } peers { permanent_uuid: "df9781e00a36431b928f661b7fcb5f27" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44913 } } peers { permanent_uuid: "38ddf18e227c464db09058d168ecd02a" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 36977 } }
I20260812 14:18:08.121097 14494 leader_election.cc:288] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d33f3ef709c04145b05e6bf84a947af9 (127.8.85.1:41779), df9781e00a36431b928f661b7fcb5f27 (127.8.85.3:44913)
I20260812 14:18:08.122558 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:08.121413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:56807 (local address 127.8.85.1:41779)
0812 14:18:08.121485 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:08.121487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:08.121661 (+ 174us) server_negotiation.cc:407] Connection header received
0812 14:18:08.121691 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:08.121692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:08.121698 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:08.121763 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:08.122324 (+ 561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:08.122326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:08.122326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:08.122331 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:08.122347 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:08.122352 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:08.122430 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:08.122474 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 14:18:08.122490 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:18:08.122577 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:08.121333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:41779 (local address 127.8.85.2:56807)
0812 14:18:08.121581 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:08.121590 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:08.121642 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:08.121772 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:08.121773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:08.121776 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:08.122223 (+ 447us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:18:08.122223 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:08.122230 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:18:08.122241 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:08.122426 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:08.122427 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:08.122428 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 14:18:08.122476 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 14:18:08.122490 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":64,"thread_start_us":112,"threads_started":1}
I20260812 14:18:08.122756 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:08.121333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:44913 (local address 127.8.85.2:57809)
0812 14:18:08.121433 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:08.121445 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:08.121510 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:08.121817 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:08.121818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:08.121820 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:08.122391 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:18:08.122392 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:08.122405 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:18:08.122417 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:08.122608 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:08.122609 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:08.122609 (+ 0us) client_negotiation.cc:769] Sending connection context
0812 14:18:08.122659 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 14:18:08.122673 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 14:18:08.122773 14482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:08.121495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:57809 (local address 127.8.85.3:44913)
0812 14:18:08.121545 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:08.121547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:08.121557 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:18:08.121587 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:08.121588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:08.121594 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:08.121676 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:08.122510 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:08.122512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:08.122512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:08.122517 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:08.122539 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:08.122543 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:08.122600 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:08.122683 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 14:18:08.122700 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:18:08.122951 14109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9f93cb5c8d9444fa98e7900f863daa9" candidate_uuid: "38ddf18e227c464db09058d168ecd02a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d33f3ef709c04145b05e6bf84a947af9" is_pre_election: true
I20260812 14:18:08.123147 14109 raft_consensus.cc:2454] T c9f93cb5c8d9444fa98e7900f863daa9 P d33f3ef709c04145b05e6bf84a947af9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38ddf18e227c464db09058d168ecd02a in term 0.
I20260812 14:18:08.123171 14417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9f93cb5c8d9444fa98e7900f863daa9" candidate_uuid: "38ddf18e227c464db09058d168ecd02a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df9781e00a36431b928f661b7fcb5f27" is_pre_election: true
I20260812 14:18:08.123306 14417 raft_consensus.cc:2454] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38ddf18e227c464db09058d168ecd02a in term 0.
I20260812 14:18:08.123375 14188 leader_election.cc:302] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38ddf18e227c464db09058d168ecd02a, d33f3ef709c04145b05e6bf84a947af9; no voters:
I20260812 14:18:08.123534 14494 raft_consensus.cc:2781] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:18:08.123603 14494 raft_consensus.cc:482] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:18:08.123628 14494 raft_consensus.cc:3037] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:08.124318 14494 raft_consensus.cc:504] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33f3ef709c04145b05e6bf84a947af9" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 41779 } } peers { permanent_uuid: "df9781e00a36431b928f661b7fcb5f27" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44913 } } peers { permanent_uuid: "38ddf18e227c464db09058d168ecd02a" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 36977 } }
I20260812 14:18:08.124464 14494 leader_election.cc:288] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [CANDIDATE]: Term 1 election: Requested vote from peers d33f3ef709c04145b05e6bf84a947af9 (127.8.85.1:41779), df9781e00a36431b928f661b7fcb5f27 (127.8.85.3:44913)
I20260812 14:18:08.124584 14109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9f93cb5c8d9444fa98e7900f863daa9" candidate_uuid: "38ddf18e227c464db09058d168ecd02a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d33f3ef709c04145b05e6bf84a947af9"
I20260812 14:18:08.124614 14417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9f93cb5c8d9444fa98e7900f863daa9" candidate_uuid: "38ddf18e227c464db09058d168ecd02a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df9781e00a36431b928f661b7fcb5f27"
I20260812 14:18:08.124671 14109 raft_consensus.cc:3037] T c9f93cb5c8d9444fa98e7900f863daa9 P d33f3ef709c04145b05e6bf84a947af9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:08.124708 14417 raft_consensus.cc:3037] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:08.125427 14109 raft_consensus.cc:2454] T c9f93cb5c8d9444fa98e7900f863daa9 P d33f3ef709c04145b05e6bf84a947af9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38ddf18e227c464db09058d168ecd02a in term 1.
I20260812 14:18:08.125577 14417 raft_consensus.cc:2454] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38ddf18e227c464db09058d168ecd02a in term 1.
I20260812 14:18:08.125618 14188 leader_election.cc:302] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38ddf18e227c464db09058d168ecd02a, d33f3ef709c04145b05e6bf84a947af9; no voters:
I20260812 14:18:08.125713 14494 raft_consensus.cc:2781] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:08.125874 14494 raft_consensus.cc:686] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [term 1 LEADER]: Becoming Leader. State: Replica: 38ddf18e227c464db09058d168ecd02a, State: Running, Role: LEADER
I20260812 14:18:08.125995 14494 consensus_queue.cc:237] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33f3ef709c04145b05e6bf84a947af9" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 41779 } } peers { permanent_uuid: "df9781e00a36431b928f661b7fcb5f27" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44913 } } peers { permanent_uuid: "38ddf18e227c464db09058d168ecd02a" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 36977 } }
I20260812 14:18:08.126832 13934 catalog_manager.cc:5495] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a reported cstate change: term changed from 0 to 1, leader changed from <none> to 38ddf18e227c464db09058d168ecd02a (127.8.85.2). New cstate: current_term: 1 leader_uuid: "38ddf18e227c464db09058d168ecd02a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33f3ef709c04145b05e6bf84a947af9" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 41779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df9781e00a36431b928f661b7fcb5f27" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38ddf18e227c464db09058d168ecd02a" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 36977 } health_report { overall_health: HEALTHY } } }
I20260812 14:18:08.131191 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:08.130210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37602 (local address 127.8.85.2:36977)
0812 14:18:08.130268 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:08.130270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:08.130277 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:18:08.130304 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:08.130305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:08.130315 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:08.130364 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:08.130913 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:08.130914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:08.130915 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:08.130920 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:08.130935 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:08.130939 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:08.130989 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:08.131077 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 14:18:08.131107 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:18:08.132822 14417 raft_consensus.cc:1261] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27 [term 1 FOLLOWER]: Refusing update from remote peer 38ddf18e227c464db09058d168ecd02a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:08.132826 14109 raft_consensus.cc:1261] T c9f93cb5c8d9444fa98e7900f863daa9 P d33f3ef709c04145b05e6bf84a947af9 [term 1 FOLLOWER]: Refusing update from remote peer 38ddf18e227c464db09058d168ecd02a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:08.133131 14496 consensus_queue.cc:1035] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [LEADER]: Connected to new peer: Peer: permanent_uuid: "df9781e00a36431b928f661b7fcb5f27" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:08.133272 14494 consensus_queue.cc:1035] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d33f3ef709c04145b05e6bf84a947af9" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 41779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:08.135771 14502 mvcc.cc:205] Tried to move back new op lower bound from 7317685404189679616 to 7317685404164571136. Current Snapshot: MvccSnapshot[applied={T|T < 7317685404189679616}]
I20260812 14:18:08.136487 14500 mvcc.cc:205] Tried to move back new op lower bound from 7317685404189679616 to 7317685404164571136. Current Snapshot: MvccSnapshot[applied={T|T < 7317685404189679616 or (T in {7317685404189679616})}]
I20260812 14:18:08.137674 13933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:08.137790 13933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:08.138582 13933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ed08ee1ca1640a2a4ae3dcde8587332: Sending DeleteTablet for 3 replicas of tablet c9f93cb5c8d9444fa98e7900f863daa9
I20260812 14:18:08.138964 14083 tablet_service.cc:1514] Processing DeleteTablet for tablet c9f93cb5c8d9444fa98e7900f863daa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:08 UTC) from {username='slave'} at 127.0.0.1:57172
I20260812 14:18:08.139082 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet c9f93cb5c8d9444fa98e7900f863daa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:08 UTC) from {username='slave'} at 127.0.0.1:41238
I20260812 14:18:08.139199 14245 tablet_service.cc:1514] Processing DeleteTablet for tablet c9f93cb5c8d9444fa98e7900f863daa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:08 UTC) from {username='slave'} at 127.0.0.1:37586
I20260812 14:18:08.139338 14513 tablet_replica.cc:330] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27: stopping tablet replica
I20260812 14:18:08.139456 14514 tablet_replica.cc:330] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a: stopping tablet replica
I20260812 14:18:08.139559 14514 raft_consensus.cc:2229] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:18:08.139566 14513 raft_consensus.cc:2229] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:08.139657 14514 raft_consensus.cc:2258] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:08.139647 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 14013
I20260812 14:18:08.139786 14513 raft_consensus.cc:2258] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:08.140144 14514 ts_tablet_manager.cc:1905] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:08.140318 14513 ts_tablet_manager.cc:1905] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:08.142196 14513 ts_tablet_manager.cc:1918] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:08.142333 14513 log.cc:1196] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/c9f93cb5c8d9444fa98e7900f863daa9
I20260812 14:18:08.142325 14514 ts_tablet_manager.cc:1918] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:08.142403 14514 log.cc:1196] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/c9f93cb5c8d9444fa98e7900f863daa9
I20260812 14:18:08.142750 14513 ts_tablet_manager.cc:1939] T c9f93cb5c8d9444fa98e7900f863daa9 P df9781e00a36431b928f661b7fcb5f27: Deleting consensus metadata
I20260812 14:18:08.142759 14514 ts_tablet_manager.cc:1939] T c9f93cb5c8d9444fa98e7900f863daa9 P 38ddf18e227c464db09058d168ecd02a: Deleting consensus metadata
I20260812 14:18:08.143330 13918 catalog_manager.cc:4841] TS 38ddf18e227c464db09058d168ecd02a (127.8.85.2:36977): tablet c9f93cb5c8d9444fa98e7900f863daa9 (table test-table [id=c640a64119d14a05b0e7a17f581fff4b]) successfully deleted
I20260812 14:18:08.151613 13917 catalog_manager.cc:4841] TS df9781e00a36431b928f661b7fcb5f27 (127.8.85.3:44913): tablet c9f93cb5c8d9444fa98e7900f863daa9 (table test-table [id=c640a64119d14a05b0e7a17f581fff4b]) successfully deleted
W20260812 14:18:08.156260 13918 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:41779 (error 108)
I20260812 14:18:08.156826 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:08.156677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:41779 (local address 127.0.0.1:57174)
0812 14:18:08.156740 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:08.156788 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:41779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 14:18:08.156909 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 14167
W20260812 14:18:08.160115 13918 catalog_manager.cc:4584] TS d33f3ef709c04145b05e6bf84a947af9 (127.8.85.1:41779): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c9f93cb5c8d9444fa98e7900f863daa9: Network error: Client connection negotiation failed: client connection to 127.8.85.1:41779: connect: Connection refused (error 111)
I20260812 14:18:08.164090 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 14326
I20260812 14:18:08.173810 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 13895
2026-08-12T14:18:08Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (865 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 14:18:08.195956 8532 mini_oidc.cc:152] Starting JWKS server
I20260812 14:18:08.197142 8532 webserver.cc:460] Webserver started at https://127.0.0.1:40977/ using document root <none> and password file <none>
I20260812 14:18:08.197227 8532 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 14:18:08.197366 8532 webserver.cc:460] Webserver started at http://127.0.0.1:33229/ using document root <none> and password file <none>
2026-08-12T14:18:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:08Z Disabled control of system clock
I20260812 14:18:08.206441 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:43847
--webserver_interface=127.8.85.62
--webserver_port=0
--builtin_ntp_servers=127.8.85.20:34637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:43847
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40977/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/testchainca.cert with env {}
W20260812 14:18:08.293068 14523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:08.293267 14523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:08.293289 14523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:08.294405 14523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:08.294440 14523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:08.294453 14523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:08.294466 14523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:08.294479 14523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 14:18:08.294492 14523 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 14:18:08.294503 14523 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40977/jwks.json
I20260812 14:18:08.296098 14523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:34637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:43847
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:43847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40977/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:08.296293 14523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:08.296450 14523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:08.301182 14531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:08.301215 14529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:08.301205 14528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:08.301492 14523 server_base.cc:1034] running on GCE node
I20260812 14:18:08.301755 14523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:08.301999 14523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:08.303196 14523 hybrid_clock.cc:648] HybridClock initialized: now 1786544288303186 us; error 37 us; skew 500 ppm
I20260812 14:18:08.304629 14523 webserver.cc:460] Webserver started at http://127.8.85.62:32813/ using document root <none> and password file <none>
I20260812 14:18:08.304921 14523 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:08.304994 14523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:08.305140 14523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:08.306198 14523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "0200115cb9bc4791afb266fc48cff04c"
format_stamp: "Formatted at 2026-08-12 14:18:08 on dist-test-slave-5k54"
server_key: "3abe247350e9f711b173ba10d8fd669d"
server_key_iv: "c33ea9766f1d7f7fc2f0b99ed3126273"
server_key_version: "encryptionkey@0"
I20260812 14:18:08.306576 14523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "0200115cb9bc4791afb266fc48cff04c"
format_stamp: "Formatted at 2026-08-12 14:18:08 on dist-test-slave-5k54"
server_key: "3abe247350e9f711b173ba10d8fd669d"
server_key_iv: "c33ea9766f1d7f7fc2f0b99ed3126273"
server_key_version: "encryptionkey@0"
I20260812 14:18:08.308126 14523 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 14:18:08.308828 14537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:08.309027 14523 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:08.309121 14523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "0200115cb9bc4791afb266fc48cff04c"
format_stamp: "Formatted at 2026-08-12 14:18:08 on dist-test-slave-5k54"
server_key: "3abe247350e9f711b173ba10d8fd669d"
server_key_iv: "c33ea9766f1d7f7fc2f0b99ed3126273"
server_key_version: "encryptionkey@0"
I20260812 14:18:08.309206 14523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:08.322767 14523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:08.323112 14523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:08.323271 14523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:08.327314 14523 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:43847
I20260812 14:18:08.327373 14594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:43847 every 8 connection(s)
I20260812 14:18:08.327780 14523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:08.328830 14595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:08.330962 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 14523
I20260812 14:18:08.331059 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:08.331298 8532 external_mini_cluster.cc:1428] Setting key 10940e597ac3dd3b9b59903af2d74cb7
I20260812 14:18:08.331344 14595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c: Bootstrap starting.
I20260812 14:18:08.332168 14595 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:08.332489 14595 log.cc:824] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:08.333209 14595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c: No bootstrap required, opened a new log
I20260812 14:18:08.335055 14595 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0200115cb9bc4791afb266fc48cff04c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43847 } }
I20260812 14:18:08.335322 14595 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:08.335400 14595 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0200115cb9bc4791afb266fc48cff04c, State: Initialized, Role: FOLLOWER
I20260812 14:18:08.335572 14595 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0200115cb9bc4791afb266fc48cff04c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43847 } }
I20260812 14:18:08.335688 14595 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:08.335747 14595 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:08.335801 14595 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:08.336367 14595 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0200115cb9bc4791afb266fc48cff04c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43847 } }
I20260812 14:18:08.336491 14595 leader_election.cc:302] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0200115cb9bc4791afb266fc48cff04c; no voters:
I20260812 14:18:08.336658 14595 leader_election.cc:288] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:08.336692 14600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:08.336817 14600 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [term 1 LEADER]: Becoming Leader. State: Replica: 0200115cb9bc4791afb266fc48cff04c, State: Running, Role: LEADER
I20260812 14:18:08.336954 14595 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:08.336959 14600 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0200115cb9bc4791afb266fc48cff04c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43847 } }
I20260812 14:18:08.337152 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:08.331958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34912 (local address 127.8.85.62:43847)
0812 14:18:08.332161 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:08.332169 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:08.332194 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:18:08.332727 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:08.332736 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:08.332816 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:08.332974 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:08.333188 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.334127 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.334938 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.335410 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.335609 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:08.336316 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:08.336320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:08.336320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:08.336536 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:08.336587 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:08.336602 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:08.336667 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:08.336792 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 14:18:08.336891 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260812 14:18:08.337425 14601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0200115cb9bc4791afb266fc48cff04c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0200115cb9bc4791afb266fc48cff04c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43847 } } }
I20260812 14:18:08.337414 14600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0200115cb9bc4791afb266fc48cff04c. Latest consensus state: current_term: 1 leader_uuid: "0200115cb9bc4791afb266fc48cff04c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0200115cb9bc4791afb266fc48cff04c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43847 } } }
I20260812 14:18:08.337651 14600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:08.337852 14601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:08.337998 14610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:08.338267 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:18:08.338585 14610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:08.340416 14610 catalog_manager.cc:1347] Generated new cluster ID: 1458a048683949ed9c3273b99a7d8588
I20260812 14:18:08.340471 14610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:18:08.350220 14610 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:08.350849 14610 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:08.358408 14610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0200115cb9bc4791afb266fc48cff04c: Generated new TSK 0
I20260812 14:18:08.358639 14610 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 14:18:08.389194 14628 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 14:18:08.389380 14628 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260812 14:18:08.389926 14627 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:08.373874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:43847 (local address 127.0.0.1:34924)
0812 14:18:08.374101 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:08.374111 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:08.374190 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:08.374388 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:08.374388 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:08.374390 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 14:18:08.374476 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:08.374478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.375203 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.375204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:08.375874 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:08.375876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.376251 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.376254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:08.376336 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:08.376354 (+ 18us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 14:18:08.389507 (+ 13153us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40977/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 14:18:08.389609 (+ 102us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.85.62:43847: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40977/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":105,"threads_started":1}
I20260812 14:18:08.390257 14598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:08.373942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34924 (local address 127.8.85.62:43847)
0812 14:18:08.374016 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:08.374018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:08.374167 (+ 149us) server_negotiation.cc:407] Connection header received
0812 14:18:08.374281 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:08.374282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:08.374295 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:08.374379 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0812 14:18:08.374557 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.375118 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.375964 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.376175 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.376253 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:08.376487 (+ 234us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 14:18:08.389315 (+ 12828us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40977/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 14:18:08.390182 (+ 867us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34924: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40977/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":44}
W20260812 14:18:08.390332 14598 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34924: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40977/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.85.62:43847: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40977/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 14:18:08.390695 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 14523
2026-08-12T14:18:08Z chronyd exiting
I20260812 14:18:08.430589 8532 test_util.cc:175] -----------------------------------------------
I20260812 14:18:08.430680 8532 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786544263511571-8532-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (236 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 14:18:08.431797 8532 mini_oidc.cc:152] Starting JWKS server
I20260812 14:18:08.432482 8532 webserver.cc:460] Webserver started at https://127.0.0.1:44347/ using document root <none> and password file <none>
I20260812 14:18:08.432524 8532 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 14:18:08.432636 8532 webserver.cc:460] Webserver started at http://127.0.0.1:39211/ using document root <none> and password file <none>
2026-08-12T14:18:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:08Z Disabled control of system clock
I20260812 14:18:08.442257 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:38979
--webserver_interface=127.8.85.62
--webserver_port=0
--builtin_ntp_servers=127.8.85.20:38291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:38979
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44347/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/testchainca.cert with env {}
W20260812 14:18:08.528553 14634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:08.528762 14634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:08.528785 14634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:08.530028 14634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:08.530076 14634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:08.530089 14634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:08.530131 14634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:08.530160 14634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 14:18:08.530177 14634 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 14:18:08.530201 14634 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44347/jwks.json
I20260812 14:18:08.531977 14634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:38291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:38979
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:38979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44347/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:08.532256 14634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:08.532441 14634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:08.537379 14644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:08.537447 14646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:08.537590 14634 server_base.cc:1034] running on GCE node
W20260812 14:18:08.537379 14643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:08.537932 14634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:08.538224 14634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:08.539492 14634 hybrid_clock.cc:648] HybridClock initialized: now 1786544288539402 us; error 41 us; skew 500 ppm
I20260812 14:18:08.540859 14634 webserver.cc:460] Webserver started at http://127.8.85.62:38163/ using document root <none> and password file <none>
I20260812 14:18:08.541100 14634 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:08.541169 14634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:08.541335 14634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:08.542438 14634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "f6903b84b21745858e3adc99eebeba18"
format_stamp: "Formatted at 2026-08-12 14:18:08 on dist-test-slave-5k54"
server_key: "1a1d24fb6f17b5afb1c16a2ce63ecd2a"
server_key_iv: "039148000a1ab7bde8a88f6aacf38454"
server_key_version: "encryptionkey@0"
I20260812 14:18:08.542882 14634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "f6903b84b21745858e3adc99eebeba18"
format_stamp: "Formatted at 2026-08-12 14:18:08 on dist-test-slave-5k54"
server_key: "1a1d24fb6f17b5afb1c16a2ce63ecd2a"
server_key_iv: "039148000a1ab7bde8a88f6aacf38454"
server_key_version: "encryptionkey@0"
I20260812 14:18:08.544414 14634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:18:08.545306 14653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:08.545567 14634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:08.545677 14634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "f6903b84b21745858e3adc99eebeba18"
format_stamp: "Formatted at 2026-08-12 14:18:08 on dist-test-slave-5k54"
server_key: "1a1d24fb6f17b5afb1c16a2ce63ecd2a"
server_key_iv: "039148000a1ab7bde8a88f6aacf38454"
server_key_version: "encryptionkey@0"
I20260812 14:18:08.545750 14634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:08.556843 14634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:08.557135 14634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:08.557287 14634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:08.561313 14634 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:38979
I20260812 14:18:08.561342 14708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:38979 every 8 connection(s)
I20260812 14:18:08.561692 14634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:08.562819 14709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:08.565518 14709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18: Bootstrap starting.
I20260812 14:18:08.566249 14709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:08.566584 14709 log.cc:824] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:08.567363 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 14634
I20260812 14:18:08.567512 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:08.567510 14709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18: No bootstrap required, opened a new log
I20260812 14:18:08.567828 8532 external_mini_cluster.cc:1428] Setting key 30370ed1453d9f859beb4006cc14e700
I20260812 14:18:08.569864 14709 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6903b84b21745858e3adc99eebeba18" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38979 } }
I20260812 14:18:08.570233 14709 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:08.570344 14709 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6903b84b21745858e3adc99eebeba18, State: Initialized, Role: FOLLOWER
I20260812 14:18:08.570492 14709 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6903b84b21745858e3adc99eebeba18" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38979 } }
I20260812 14:18:08.570602 14709 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:08.570640 14709 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:08.570701 14709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:08.571568 14709 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6903b84b21745858e3adc99eebeba18" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38979 } }
I20260812 14:18:08.571691 14709 leader_election.cc:302] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6903b84b21745858e3adc99eebeba18; no voters:
I20260812 14:18:08.571867 14709 leader_election.cc:288] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:08.571926 14715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:08.572088 14715 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [term 1 LEADER]: Becoming Leader. State: Replica: f6903b84b21745858e3adc99eebeba18, State: Running, Role: LEADER
I20260812 14:18:08.572127 14709 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:08.572187 14715 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6903b84b21745858e3adc99eebeba18" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38979 } }
I20260812 14:18:08.572508 14715 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6903b84b21745858e3adc99eebeba18. Latest consensus state: current_term: 1 leader_uuid: "f6903b84b21745858e3adc99eebeba18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6903b84b21745858e3adc99eebeba18" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38979 } } }
I20260812 14:18:08.572609 14715 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:08.572520 14717 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6903b84b21745858e3adc99eebeba18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6903b84b21745858e3adc99eebeba18" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38979 } } }
I20260812 14:18:08.572773 14717 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:08.572896 14722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:08.573398 14722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:08.575851 14722 catalog_manager.cc:1347] Generated new cluster ID: bc160cfe160440979165e75853af01c3
I20260812 14:18:08.575922 14722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:18:08.576411 14714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:08.568401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48950 (local address 127.8.85.62:38979)
0812 14:18:08.568639 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:08.568649 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:08.568689 (+ 40us) server_negotiation.cc:407] Connection header received
0812 14:18:08.569815 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:08.569825 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:08.569921 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:08.570109 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:08.570355 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.572385 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.573796 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.574509 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.574639 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:08.575511 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:08.575515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:08.575516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:08.575725 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:08.575770 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:08.575785 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:08.575859 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:08.576098 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 14:18:08.576188 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":81,"threads_started":1}
I20260812 14:18:08.577440 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:18:08.594947 14722 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:08.595602 14722 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:08.598850 14722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6903b84b21745858e3adc99eebeba18: Generated new TSK 0
I20260812 14:18:08.599049 14722 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 14:18:08.622102 14743 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 14:18:08.622196 14743 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 14:18:08.622500 14714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:08.608193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48954 (local address 127.8.85.62:38979)
0812 14:18:08.608270 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:08.608272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:08.608359 (+ 87us) server_negotiation.cc:407] Connection header received
0812 14:18:08.608447 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:08.608448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:08.608459 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:08.608520 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0812 14:18:08.608695 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.609167 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.610316 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.610512 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.610568 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:08.610771 (+ 203us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 14:18:08.622330 (+ 11559us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44347/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 14:18:08.622453 (+ 123us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48954: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44347/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 14:18:08.622572 14714 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48954: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44347/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 14:18:08.622669 14742 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:08.608127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38979 (local address 127.0.0.1:48954)
0812 14:18:08.608291 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:08.608303 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:08.608371 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:08.608542 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:08.608543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:08.608545 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 14:18:08.608623 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:08.608625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.609246 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.609247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:08.610223 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:08.610226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.610589 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.610593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:08.610665 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:08.610671 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 14:18:08.622447 (+ 11776us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44347/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 14:18:08.622566 (+ 119us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.85.62:38979: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44347/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":109,"threads_started":1}
I20260812 14:18:08.623005 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 14634
2026-08-12T14:18:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (236 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 14:18:08.668203 8532 mini_oidc.cc:152] Starting JWKS server
I20260812 14:18:08.668987 8532 webserver.cc:460] Webserver started at https://127.0.0.1:38225/ using document root <none> and password file <none>
I20260812 14:18:08.669063 8532 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 14:18:08.669231 8532 webserver.cc:460] Webserver started at http://127.0.0.1:45667/ using document root <none> and password file <none>
2026-08-12T14:18:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:08Z Disabled control of system clock
I20260812 14:18:08.678411 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:42995
--webserver_interface=127.8.85.62
--webserver_port=0
--builtin_ntp_servers=127.8.85.20:42405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:42995
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38225/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 14:18:08.760794 14750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:08.760995 14750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:08.761018 14750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:08.762102 14750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:08.762136 14750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:08.762183 14750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:08.762216 14750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:08.762238 14750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 14:18:08.762264 14750 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 14:18:08.762282 14750 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38225/jwks.json
I20260812 14:18:08.763665 14750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:42405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:42995
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:42995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38225/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:08.763872 14750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:08.764124 14750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:08.768715 14759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:08.768716 14756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:08.768958 14750 server_base.cc:1034] running on GCE node
W20260812 14:18:08.768700 14755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:08.769243 14750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:08.769536 14750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:08.770702 14750 hybrid_clock.cc:648] HybridClock initialized: now 1786544288770688 us; error 43 us; skew 500 ppm
I20260812 14:18:08.771976 14750 webserver.cc:460] Webserver started at http://127.8.85.62:36687/ using document root <none> and password file <none>
I20260812 14:18:08.772218 14750 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:08.772285 14750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:08.772454 14750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:08.773465 14750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "632060289dcd428280f28009824340cb"
format_stamp: "Formatted at 2026-08-12 14:18:08 on dist-test-slave-5k54"
server_key: "577c157f68f76fdfd6ccd77c8be5977c"
server_key_iv: "9a2625d77335070ca55c32a1622b7bdb"
server_key_version: "encryptionkey@0"
I20260812 14:18:08.773897 14750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "632060289dcd428280f28009824340cb"
format_stamp: "Formatted at 2026-08-12 14:18:08 on dist-test-slave-5k54"
server_key: "577c157f68f76fdfd6ccd77c8be5977c"
server_key_iv: "9a2625d77335070ca55c32a1622b7bdb"
server_key_version: "encryptionkey@0"
I20260812 14:18:08.775218 14750 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:18:08.775995 14767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:08.776162 14750 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:08.776281 14750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "632060289dcd428280f28009824340cb"
format_stamp: "Formatted at 2026-08-12 14:18:08 on dist-test-slave-5k54"
server_key: "577c157f68f76fdfd6ccd77c8be5977c"
server_key_iv: "9a2625d77335070ca55c32a1622b7bdb"
server_key_version: "encryptionkey@0"
I20260812 14:18:08.776358 14750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:08.784502 14750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:08.784791 14750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:08.784955 14750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:08.788875 14750 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:42995
I20260812 14:18:08.788935 14830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:42995 every 8 connection(s)
I20260812 14:18:08.789285 14750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:08.790007 14831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:08.792665 14831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb: Bootstrap starting.
I20260812 14:18:08.792910 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 14750
I20260812 14:18:08.793023 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:08.793237 8532 external_mini_cluster.cc:1428] Setting key 7d563f5542dd45f5fce6fd56a1cfbd56
I20260812 14:18:08.793562 14831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:08.794099 14831 log.cc:824] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:08.794800 14831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb: No bootstrap required, opened a new log
I20260812 14:18:08.796599 14831 raft_consensus.cc:348] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632060289dcd428280f28009824340cb" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42995 } }
I20260812 14:18:08.796764 14831 raft_consensus.cc:374] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:08.796815 14831 raft_consensus.cc:729] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632060289dcd428280f28009824340cb, State: Initialized, Role: FOLLOWER
I20260812 14:18:08.796972 14831 consensus_queue.cc:260] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632060289dcd428280f28009824340cb" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42995 } }
I20260812 14:18:08.797071 14831 raft_consensus.cc:388] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:08.797117 14831 raft_consensus.cc:482] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:08.797168 14831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:08.797727 14831 raft_consensus.cc:504] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632060289dcd428280f28009824340cb" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42995 } }
I20260812 14:18:08.797840 14831 leader_election.cc:302] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 632060289dcd428280f28009824340cb; no voters:
I20260812 14:18:08.798050 14831 leader_election.cc:288] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:08.798074 14837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:08.798194 14837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [term 1 LEADER]: Becoming Leader. State: Replica: 632060289dcd428280f28009824340cb, State: Running, Role: LEADER
I20260812 14:18:08.798285 14837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632060289dcd428280f28009824340cb" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42995 } }
I20260812 14:18:08.798344 14831 sys_catalog.cc:564] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:08.798610 14838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "632060289dcd428280f28009824340cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632060289dcd428280f28009824340cb" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42995 } } }
I20260812 14:18:08.798848 14838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:08.798781 14839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 632060289dcd428280f28009824340cb. Latest consensus state: current_term: 1 leader_uuid: "632060289dcd428280f28009824340cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632060289dcd428280f28009824340cb" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42995 } } }
I20260812 14:18:08.799095 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:08.793708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57086 (local address 127.8.85.62:42995)
0812 14:18:08.793892 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:08.793899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:08.793925 (+ 26us) server_negotiation.cc:407] Connection header received
0812 14:18:08.794391 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:08.794398 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:08.794463 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:08.794609 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:08.794845 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.795851 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.796696 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.797278 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.797418 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:08.798121 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:08.798126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:08.798126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:08.798344 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:08.798391 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:08.798406 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:08.798708 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:08.798743 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 14:18:08.798830 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260812 14:18:08.799226 14839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:08.799338 14846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:08.799957 14846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:08.800113 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:18:08.801527 14846 catalog_manager.cc:1347] Generated new cluster ID: f9aebd1191ec4d49a7692bb02411f418
I20260812 14:18:08.801585 14846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:18:08.819588 14846 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:08.820286 14846 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:08.829324 14846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 632060289dcd428280f28009824340cb: Generated new TSK 0
I20260812 14:18:08.829532 14846 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:18:08.848467 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:08.844735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57088 (local address 127.8.85.62:42995)
0812 14:18:08.844789 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:08.844793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:08.844885 (+ 92us) server_negotiation.cc:407] Connection header received
0812 14:18:08.844951 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:08.844951 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:08.844962 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:08.845021 (+ 59us) server_negotiation.cc:226] Negotiated authn=JWT
0812 14:18:08.845180 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.845659 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.846866 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.847049 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.847099 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:08.847291 (+ 192us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 14:18:08.848240 (+ 949us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38225/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 14:18:08.848372 (+ 132us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57088: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38225/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 14:18:08.848531 14834 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57088: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38225/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 14:18:08.848577 14864 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:08.844611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:42995 (local address 127.0.0.1:57088)
0812 14:18:08.844798 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:08.844810 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:08.844880 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:08.845048 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:08.845049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:08.845051 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 14:18:08.845115 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:08.845118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.845776 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.845778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:08.846781 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:08.846784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:08.847127 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:08.847131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:08.847193 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:08.847199 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 14:18:08.848353 (+ 1154us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38225/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 14:18:08.848462 (+ 109us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.85.62:42995: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38225/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":102,"threads_started":1}
I20260812 14:18:08.848803 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 14750
E20260812 14:18:08.861943 14865 webserver.cc:533] Webserver: sslize failed:
E20260812 14:18:08.862038 14865 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T14:18:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (227 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T14:18:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:08Z Disabled control of system clock
I20260812 14:18:08.904376 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:44077
--webserver_interface=127.8.85.62
--webserver_port=0
--builtin_ntp_servers=127.8.85.20:44507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:44077
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/testchainca.cert with env {}
W20260812 14:18:08.986790 14871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:08.986990 14871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:08.987012 14871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:08.988287 14871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:08.988329 14871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:08.988341 14871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:08.988354 14871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:08.988366 14871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 14:18:08.988379 14871 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 14:18:08.988410 14871 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 14:18:08.989897 14871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:44507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:44077
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:44077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:08.990140 14871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:08.990381 14871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:08.995177 14879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:08.995221 14877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:08.995378 14876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:08.995363 14871 server_base.cc:1034] running on GCE node
I20260812 14:18:08.995677 14871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:08.995913 14871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:08.997102 14871 hybrid_clock.cc:648] HybridClock initialized: now 1786544288997075 us; error 35 us; skew 500 ppm
I20260812 14:18:08.998355 14871 webserver.cc:460] Webserver started at http://127.8.85.62:42633/ using document root <none> and password file <none>
I20260812 14:18:08.998591 14871 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:08.998672 14871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:08.998803 14871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:08.999902 14871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "a461c4bb26194c3bb224391723d271c4"
format_stamp: "Formatted at 2026-08-12 14:18:08 on dist-test-slave-5k54"
server_key: "c9937db6de44e99201ef725e4dd81b47"
server_key_iv: "b9c2b7933b2d04b88a2591d6ccff1ada"
server_key_version: "encryptionkey@0"
I20260812 14:18:09.000329 14871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "a461c4bb26194c3bb224391723d271c4"
format_stamp: "Formatted at 2026-08-12 14:18:08 on dist-test-slave-5k54"
server_key: "c9937db6de44e99201ef725e4dd81b47"
server_key_iv: "b9c2b7933b2d04b88a2591d6ccff1ada"
server_key_version: "encryptionkey@0"
I20260812 14:18:09.001698 14871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:09.002542 14885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:09.002758 14871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:09.002866 14871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "a461c4bb26194c3bb224391723d271c4"
format_stamp: "Formatted at 2026-08-12 14:18:08 on dist-test-slave-5k54"
server_key: "c9937db6de44e99201ef725e4dd81b47"
server_key_iv: "b9c2b7933b2d04b88a2591d6ccff1ada"
server_key_version: "encryptionkey@0"
I20260812 14:18:09.002941 14871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:09.010891 14871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:09.011164 14871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:09.011312 14871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:09.015277 14871 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:44077
I20260812 14:18:09.015323 14950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:44077 every 8 connection(s)
I20260812 14:18:09.015699 14871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:09.016633 14954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:09.018739 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 14871
I20260812 14:18:09.018810 14954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4: Bootstrap starting.
I20260812 14:18:09.018886 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:09.019104 8532 external_mini_cluster.cc:1428] Setting key e3b9579cf46ec3b82bc5587467f2316d
I20260812 14:18:09.019580 14954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:09.019903 14954 log.cc:824] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:09.020609 14954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4: No bootstrap required, opened a new log
I20260812 14:18:09.022418 14954 raft_consensus.cc:348] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a461c4bb26194c3bb224391723d271c4" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44077 } }
I20260812 14:18:09.022576 14954 raft_consensus.cc:374] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:09.022625 14954 raft_consensus.cc:729] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a461c4bb26194c3bb224391723d271c4, State: Initialized, Role: FOLLOWER
I20260812 14:18:09.022779 14954 consensus_queue.cc:260] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a461c4bb26194c3bb224391723d271c4" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44077 } }
I20260812 14:18:09.022881 14954 raft_consensus.cc:388] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:09.022926 14954 raft_consensus.cc:482] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:09.022974 14954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:09.023574 14954 raft_consensus.cc:504] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a461c4bb26194c3bb224391723d271c4" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44077 } }
I20260812 14:18:09.023690 14954 leader_election.cc:302] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a461c4bb26194c3bb224391723d271c4; no voters:
I20260812 14:18:09.023851 14954 leader_election.cc:288] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:09.023885 14959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:09.024051 14959 raft_consensus.cc:686] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [term 1 LEADER]: Becoming Leader. State: Replica: a461c4bb26194c3bb224391723d271c4, State: Running, Role: LEADER
I20260812 14:18:09.024117 14954 sys_catalog.cc:564] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:09.024144 14959 consensus_queue.cc:237] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a461c4bb26194c3bb224391723d271c4" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44077 } }
I20260812 14:18:09.024497 14959 sys_catalog.cc:455] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a461c4bb26194c3bb224391723d271c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a461c4bb26194c3bb224391723d271c4" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44077 } } }
I20260812 14:18:09.024573 14959 sys_catalog.cc:458] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:09.024730 14960 sys_catalog.cc:455] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a461c4bb26194c3bb224391723d271c4. Latest consensus state: current_term: 1 leader_uuid: "a461c4bb26194c3bb224391723d271c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a461c4bb26194c3bb224391723d271c4" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44077 } } }
I20260812 14:18:09.024823 14960 sys_catalog.cc:458] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:09.025660 14957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:09.019718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44376 (local address 127.8.85.62:44077)
0812 14:18:09.019998 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:09.020010 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:09.020079 (+ 69us) server_negotiation.cc:407] Connection header received
0812 14:18:09.020633 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:09.020640 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:09.020733 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:09.020904 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:09.021163 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:09.022125 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:09.023245 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:09.023889 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:09.024080 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:09.024816 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:09.024821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:09.024821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:09.025069 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:09.025122 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:09.025138 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:09.025213 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:09.025345 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 14:18:09.025463 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":142,"threads_started":1}
I20260812 14:18:09.025955 14975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:09.026569 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:18:09.027556 14975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:09.029099 14975 catalog_manager.cc:1347] Generated new cluster ID: 0df878be56f64cb19645e8767ba05da9
I20260812 14:18:09.029175 14975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:18:09.047923 14975 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:09.048491 14975 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:09.060755 14975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a461c4bb26194c3bb224391723d271c4: Generated new TSK 0
I20260812 14:18:09.060986 14975 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:18:09.519850 14957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:09.067308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44384 (local address 127.8.85.62:44077)
0812 14:18:09.067357 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:09.067360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:09.067475 (+ 115us) server_negotiation.cc:407] Connection header received
0812 14:18:09.067569 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:09.067570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:09.067582 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:09.067663 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0812 14:18:09.067777 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:09.068581 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:09.069355 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:09.069675 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:09.069741 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:09.069890 (+ 149us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 14:18:09.519591 (+449701us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 14:18:09.519775 (+ 184us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44384: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":26}
W20260812 14:18:09.519923 14957 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44384: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 14:18:09.520082 14987 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:09.067218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44077 (local address 127.0.0.1:44384)
0812 14:18:09.067373 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:09.067383 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:09.067495 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:09.067669 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:09.067669 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:09.067671 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 14:18:09.067720 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:09.067722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:09.068687 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:09.068687 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:09.069271 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:09.069273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:09.069755 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:09.069758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:09.069798 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:09.069802 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 14:18:09.519754 (+449952us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 14:18:09.519900 (+ 146us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.85.62:44077: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":100,"threads_started":1}
I20260812 14:18:09.520496 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 14871
2026-08-12T14:18:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (648 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:18:09 dist-test-slave-5k54 krb5kdc[15001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:18:09 dist-test-slave-5k54 krb5kdc[15001](info): set up 2 sockets
Aug 12 14:18:09 dist-test-slave-5k54 krb5kdc[15001](info): commencing operation
krb5kdc: starting...
W20260812 14:18:10.827026 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:18:10 dist-test-slave-5k54 krb5kdc[15001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:18:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:18:10.956442 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:42101
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:43923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:42101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:11.038240 15016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:11.038476 15016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:11.038515 15016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:11.039747 15016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:11.039812 15016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:11.039836 15016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:11.039860 15016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:11.039904 15016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:11.041343 15016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:43923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:42101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:42101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:11.041617 15016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:11.041843 15016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:11.046689 15022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:11.046700 15024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:11.046700 15021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:11.047021 15016 server_base.cc:1034] running on GCE node
I20260812 14:18:11.047240 15016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:11.047535 15016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:11.048677 15016 hybrid_clock.cc:648] HybridClock initialized: now 1786544291048658 us; error 35 us; skew 500 ppm
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:11.051048 15016 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:18:11.051568 15016 webserver.cc:460] Webserver started at http://127.8.85.62:45755/ using document root <none> and password file <none>
I20260812 14:18:11.051750 15016 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:11.051790 15016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:11.051878 15016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:11.052909 15016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "ace0a59d83be4c6693c831b5b32878ba"
format_stamp: "Formatted at 2026-08-12 14:18:11 on dist-test-slave-5k54"
server_key: "16d2f7ca26b263a12427e9b00e89bd2f"
server_key_iv: "14ae44d59661c2a19a60dfe2f342d793"
server_key_version: "encryptionkey@0"
I20260812 14:18:11.053273 15016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "ace0a59d83be4c6693c831b5b32878ba"
format_stamp: "Formatted at 2026-08-12 14:18:11 on dist-test-slave-5k54"
server_key: "16d2f7ca26b263a12427e9b00e89bd2f"
server_key_iv: "14ae44d59661c2a19a60dfe2f342d793"
server_key_version: "encryptionkey@0"
I20260812 14:18:11.054569 15016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:18:11.055215 15035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:11.055387 15016 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:11.055519 15016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "ace0a59d83be4c6693c831b5b32878ba"
format_stamp: "Formatted at 2026-08-12 14:18:11 on dist-test-slave-5k54"
server_key: "16d2f7ca26b263a12427e9b00e89bd2f"
server_key_iv: "14ae44d59661c2a19a60dfe2f342d793"
server_key_version: "encryptionkey@0"
I20260812 14:18:11.055600 15016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:11.078287 15016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:11.080482 15016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:11.080655 15016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:11.084915 15016 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:42101
I20260812 14:18:11.084962 15098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:42101 every 8 connection(s)
I20260812 14:18:11.085350 15016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:11.086244 15099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:11.088560 15099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba: Bootstrap starting.
I20260812 14:18:11.089123 15099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:11.089371 15099 log.cc:824] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:11.090085 15099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba: No bootstrap required, opened a new log
I20260812 14:18:11.091082 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 15016
I20260812 14:18:11.091193 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:11.091396 8532 external_mini_cluster.cc:1428] Setting key 3cf8dde00c98498b0e0dc39a24a39705
I20260812 14:18:11.092378 15099 raft_consensus.cc:348] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace0a59d83be4c6693c831b5b32878ba" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42101 } }
I20260812 14:18:11.092571 15099 raft_consensus.cc:374] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:11.092624 15099 raft_consensus.cc:729] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ace0a59d83be4c6693c831b5b32878ba, State: Initialized, Role: FOLLOWER
I20260812 14:18:11.092762 15099 consensus_queue.cc:260] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace0a59d83be4c6693c831b5b32878ba" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42101 } }
I20260812 14:18:11.092868 15099 raft_consensus.cc:388] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:11.092902 15099 raft_consensus.cc:482] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:11.092957 15099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:11.094029 15099 raft_consensus.cc:504] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace0a59d83be4c6693c831b5b32878ba" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42101 } }
I20260812 14:18:11.094163 15099 leader_election.cc:302] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ace0a59d83be4c6693c831b5b32878ba; no voters:
I20260812 14:18:11.094362 15099 leader_election.cc:288] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:11.094399 15104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:11.094558 15104 raft_consensus.cc:686] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [term 1 LEADER]: Becoming Leader. State: Replica: ace0a59d83be4c6693c831b5b32878ba, State: Running, Role: LEADER
I20260812 14:18:11.094625 15099 sys_catalog.cc:564] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:11.094677 15104 consensus_queue.cc:237] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace0a59d83be4c6693c831b5b32878ba" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42101 } }
I20260812 14:18:11.095032 15104 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader ace0a59d83be4c6693c831b5b32878ba. Latest consensus state: current_term: 1 leader_uuid: "ace0a59d83be4c6693c831b5b32878ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace0a59d83be4c6693c831b5b32878ba" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42101 } } }
I20260812 14:18:11.095043 15105 sys_catalog.cc:455] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ace0a59d83be4c6693c831b5b32878ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ace0a59d83be4c6693c831b5b32878ba" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42101 } } }
I20260812 14:18:11.095232 15105 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:11.095513 15104 sys_catalog.cc:458] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:11.095877 15115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:11.096402 15115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:11.098075 15115 catalog_manager.cc:1347] Generated new cluster ID: 5b2eaef87112453eb29916952bd14f3a
I20260812 14:18:11.098124 15115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:11.102339 15103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:11.091979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47248 (local address 127.8.85.62:42101)
0812 14:18:11.092152 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:11.092160 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:11.092185 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:18:11.092324 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:11.092330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:11.092371 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:11.092481 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:11.093316 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.094284 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.095407 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.096137 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.096228 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:11.099026 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:11.099031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:11.099031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:11.099043 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:11.100766 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:11.101164 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:11.101169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:11.101169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:11.101278 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:11.101492 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:11.101494 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:11.101494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:11.101741 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:11.101917 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:11.102027 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 14:18:11.102169 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260812 14:18:11.104125 15115 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:11.104811 15115 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:11.122605 15115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ace0a59d83be4c6693c831b5b32878ba: Generated new TSK 0
I20260812 14:18:11.122836 15115 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:18:11.151162 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:42101
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:43923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:11.233989 15130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:11.234189 15130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:11.234213 15130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:11.235357 15130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:11.235400 15130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:11.235478 15130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:18:11.236980 15130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:43923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:42101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:11.237160 15130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:11.237332 15130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:11.242477 15138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:11.242488 15135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:11.242491 15136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:11.242764 15130 server_base.cc:1034] running on GCE node
I20260812 14:18:11.243084 15130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:11.243361 15130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:11.244508 15130 hybrid_clock.cc:648] HybridClock initialized: now 1786544291244486 us; error 37 us; skew 500 ppm
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:11.247207 15130 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:18:11.247697 15130 webserver.cc:460] Webserver started at http://127.8.85.1:36565/ using document root <none> and password file <none>
I20260812 14:18:11.247889 15130 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:11.247929 15130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:11.248039 15130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:11.249048 15130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "22a87e17f22545189a6e587f1f715a92"
format_stamp: "Formatted at 2026-08-12 14:18:11 on dist-test-slave-5k54"
server_key: "bb52e877a72a84aed97d8533f2c6b1ed"
server_key_iv: "4e69a0f27ff2d56dbfec3e4c8f2796cf"
server_key_version: "encryptionkey@0"
I20260812 14:18:11.249359 15130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "22a87e17f22545189a6e587f1f715a92"
format_stamp: "Formatted at 2026-08-12 14:18:11 on dist-test-slave-5k54"
server_key: "bb52e877a72a84aed97d8533f2c6b1ed"
server_key_iv: "4e69a0f27ff2d56dbfec3e4c8f2796cf"
server_key_version: "encryptionkey@0"
I20260812 14:18:11.250653 15130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:18:11.251387 15147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:11.251591 15130 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:11.251700 15130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "22a87e17f22545189a6e587f1f715a92"
format_stamp: "Formatted at 2026-08-12 14:18:11 on dist-test-slave-5k54"
server_key: "bb52e877a72a84aed97d8533f2c6b1ed"
server_key_iv: "4e69a0f27ff2d56dbfec3e4c8f2796cf"
server_key_version: "encryptionkey@0"
I20260812 14:18:11.251777 15130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:11.272313 15130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:11.274688 15130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:11.274871 15130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:11.275120 15130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:11.275484 15130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:11.275543 15130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:11.275580 15130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:11.275609 15130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:11.281208 15130 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:42367
I20260812 14:18:11.281247 15282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:42367 every 8 connection(s)
I20260812 14:18:11.281569 15130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:18:11.286329 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 15130
I20260812 14:18:11.286461 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:18:11.286724 8532 external_mini_cluster.cc:1428] Setting key 9178c25d8d00ae84f357af19d8ec9bc7
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:11.291234 15103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:11.282350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:53655 (local address 127.8.85.62:42101)
0812 14:18:11.282398 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:11.282400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:11.282594 (+ 194us) server_negotiation.cc:407] Connection header received
0812 14:18:11.283580 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:11.283581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:11.283591 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:11.283652 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:11.284264 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.284761 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.285711 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.285919 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.285970 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:11.288389 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:11.288391 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:11.288391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:11.288402 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:11.289810 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:11.290240 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:11.290242 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:11.290242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:11.290310 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:11.290576 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:11.290577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:11.290577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:11.290690 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:11.290762 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:11.291061 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 14:18:11.291144 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:18:11.291424 15288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:11.282293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:42101 (local address 127.8.85.1:53655)
0812 14:18:11.282468 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:11.282510 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:11.283417 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:11.283734 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:11.283737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:11.284001 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:11.284185 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:11.284189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.284857 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.284857 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:11.285635 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:11.285637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.286016 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.286017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:11.286115 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:11.286710 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:11.286722 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:11.288292 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:11.289912 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:11.289921 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:11.289923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:11.290153 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:11.290387 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:11.290388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:11.290388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:11.290500 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:11.290766 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:11.290774 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:11.290921 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 14:18:11.291024 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 14:18:11.291181 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260812 14:18:11.291924 15283 heartbeater.cc:347] Connected to a master server at 127.8.85.62:42101
I20260812 14:18:11.292034 15283 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:11.292237 15283 heartbeater.cc:511] Master 127.8.85.62:42101 requested a full tablet report, sending...
I20260812 14:18:11.292850 15057 ts_manager.cc:194] Registered new tserver with Master: 22a87e17f22545189a6e587f1f715a92 (127.8.85.1:42367)
I20260812 14:18:11.293475 15057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:53655
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:18:11.312361 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:42101
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:43923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:11.399567 15291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:11.399773 15291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:11.399798 15291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:11.400964 15291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:11.400997 15291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:11.401021 15291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:18:11.402410 15291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:43923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:42101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:11.402590 15291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:11.402755 15291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:11.407665 15297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:11.407680 15296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:11.407814 15291 server_base.cc:1034] running on GCE node
W20260812 14:18:11.407678 15299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:11.408214 15291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:11.408453 15291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:11.409605 15291 hybrid_clock.cc:648] HybridClock initialized: now 1786544291409595 us; error 29 us; skew 500 ppm
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:11.412232 15291 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:18:11.412618 15291 webserver.cc:460] Webserver started at http://127.8.85.2:44177/ using document root <none> and password file <none>
I20260812 14:18:11.412806 15291 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:11.412844 15291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:11.412938 15291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:11.413950 15291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "d011456e708f4e55b308d76fd8da1141"
format_stamp: "Formatted at 2026-08-12 14:18:11 on dist-test-slave-5k54"
server_key: "8153f3b94033a731970ca2aab2145298"
server_key_iv: "3ecc80f23d8ccf945507e2495e51f059"
server_key_version: "encryptionkey@0"
I20260812 14:18:11.414258 15291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "d011456e708f4e55b308d76fd8da1141"
format_stamp: "Formatted at 2026-08-12 14:18:11 on dist-test-slave-5k54"
server_key: "8153f3b94033a731970ca2aab2145298"
server_key_iv: "3ecc80f23d8ccf945507e2495e51f059"
server_key_version: "encryptionkey@0"
I20260812 14:18:11.415560 15291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:18:11.416278 15306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:11.416455 15291 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:11.416554 15291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "d011456e708f4e55b308d76fd8da1141"
format_stamp: "Formatted at 2026-08-12 14:18:11 on dist-test-slave-5k54"
server_key: "8153f3b94033a731970ca2aab2145298"
server_key_iv: "3ecc80f23d8ccf945507e2495e51f059"
server_key_version: "encryptionkey@0"
I20260812 14:18:11.416635 15291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:11.431212 15291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:11.433586 15291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:11.433748 15291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:11.433985 15291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:11.434294 15291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:11.434347 15291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:11.434381 15291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:11.434412 15291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:11.440225 15291 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:44113
I20260812 14:18:11.440323 15438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:44113 every 8 connection(s)
I20260812 14:18:11.440621 15291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:11.447342 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 15291
I20260812 14:18:11.447486 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:18:11.447701 8532 external_mini_cluster.cc:1428] Setting key ab79d9936a198d1bbd268880983e78b2
I20260812 14:18:11.450132 15103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:11.441329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:60303 (local address 127.8.85.62:42101)
0812 14:18:11.441410 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:11.441412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:11.441616 (+ 204us) server_negotiation.cc:407] Connection header received
0812 14:18:11.442564 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:11.442565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:11.442576 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:11.442625 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:11.443242 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.443831 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.444782 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.444976 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.445032 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:11.447082 (+ 2050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:11.447084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:11.447085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:11.447092 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:11.448641 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:11.449060 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:11.449062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:11.449062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:11.449129 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:11.449417 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:11.449418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:11.449418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:11.449593 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:11.449663 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:11.449902 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 14:18:11.449992 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 14:18:11.450141 15441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:11.441295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:42101 (local address 127.8.85.2:60303)
0812 14:18:11.441482 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:11.441526 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:11.442468 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:11.442688 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:11.442690 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:11.442942 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:11.443121 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:11.443126 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.443918 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.443919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:11.444700 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:11.444702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.445051 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.445052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:11.445151 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:11.445586 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:11.445594 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:11.446985 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:11.448729 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:11.448735 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:11.448737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:11.448983 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:11.449223 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:11.449224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:11.449224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:11.449342 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:11.449661 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:11.449665 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:11.449776 (+ 111us) client_negotiation.cc:769] Sending connection context
0812 14:18:11.449860 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 14:18:11.449987 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
I20260812 14:18:11.450665 15439 heartbeater.cc:347] Connected to a master server at 127.8.85.62:42101
I20260812 14:18:11.450773 15439 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:11.450966 15439 heartbeater.cc:511] Master 127.8.85.62:42101 requested a full tablet report, sending...
I20260812 14:18:11.451346 15057 ts_manager.cc:194] Registered new tserver with Master: d011456e708f4e55b308d76fd8da1141 (127.8.85.2:44113)
I20260812 14:18:11.451751 15057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:60303
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:18:11.474253 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:42101
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:43923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:11.557621 15444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:11.557816 15444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:11.557838 15444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:11.559032 15444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:11.559073 15444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:11.559099 15444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:18:11.560586 15444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:43923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:42101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:11.560825 15444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:11.561054 15444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:11.566035 15452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:11.566035 15454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:11.566039 15451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:11.566196 15444 server_base.cc:1034] running on GCE node
I20260812 14:18:11.566485 15444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:11.566718 15444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:11.567881 15444 hybrid_clock.cc:648] HybridClock initialized: now 1786544291567863 us; error 51 us; skew 500 ppm
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:11.570497 15444 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:18:11.570937 15444 webserver.cc:460] Webserver started at http://127.8.85.3:35795/ using document root <none> and password file <none>
I20260812 14:18:11.571161 15444 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:11.571229 15444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:11.571390 15444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:11.572430 15444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "97f41ff80c1846f78557f95352086b44"
format_stamp: "Formatted at 2026-08-12 14:18:11 on dist-test-slave-5k54"
server_key: "c91aca0bf0064f176c2a2d6ed669043f"
server_key_iv: "1cf33f5015b78851b97219a3852fa16e"
server_key_version: "encryptionkey@0"
I20260812 14:18:11.572803 15444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "97f41ff80c1846f78557f95352086b44"
format_stamp: "Formatted at 2026-08-12 14:18:11 on dist-test-slave-5k54"
server_key: "c91aca0bf0064f176c2a2d6ed669043f"
server_key_iv: "1cf33f5015b78851b97219a3852fa16e"
server_key_version: "encryptionkey@0"
I20260812 14:18:11.574107 15444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:11.574884 15463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:11.575073 15444 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:11.575191 15444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "97f41ff80c1846f78557f95352086b44"
format_stamp: "Formatted at 2026-08-12 14:18:11 on dist-test-slave-5k54"
server_key: "c91aca0bf0064f176c2a2d6ed669043f"
server_key_iv: "1cf33f5015b78851b97219a3852fa16e"
server_key_version: "encryptionkey@0"
I20260812 14:18:11.575284 15444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:11.598148 15444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:11.600580 15444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:11.600759 15444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:11.601022 15444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:11.601346 15444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:11.601408 15444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:11.601454 15444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:11.601477 15444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:11.607786 15444 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:35663
I20260812 14:18:11.607846 15591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:35663 every 8 connection(s)
I20260812 14:18:11.608202 15444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:18:11.609458 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 15444
I20260812 14:18:11.609535 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:18:11.609735 8532 external_mini_cluster.cc:1428] Setting key e330e021da2c653d46000744fc432e15
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:11.617641 15103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:11.608967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:56591 (local address 127.8.85.62:42101)
0812 14:18:11.609053 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:11.609056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:11.609233 (+ 177us) server_negotiation.cc:407] Connection header received
0812 14:18:11.610482 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:11.610484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:11.610494 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:11.610545 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:11.611260 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.611832 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.612791 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.612977 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.613023 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:11.615266 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:11.615268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:11.615268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:11.615274 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:11.616505 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:11.616854 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:11.616855 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:11.616855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:11.616904 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:11.617133 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:11.617134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:11.617134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:11.617236 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:11.617285 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:11.617521 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 14:18:11.617579 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 14:18:11.617846 15596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:11.608916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:42101 (local address 127.8.85.3:56591)
0812 14:18:11.609110 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:11.609146 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:11.610351 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:11.610600 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:11.610603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:11.610980 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:11.611182 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:11.611188 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.611916 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.611917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:11.612720 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:11.612722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.613043 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.613043 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:11.613140 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:11.613638 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:11.613646 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:11.615180 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:11.616576 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:11.616582 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:11.616584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:11.616790 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:11.616967 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:11.616967 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:11.616968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:11.617062 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:11.617290 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:11.617295 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:11.617407 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 14:18:11.617492 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 14:18:11.617602 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":104,"threads_started":1}
I20260812 14:18:11.618403 15594 heartbeater.cc:347] Connected to a master server at 127.8.85.62:42101
I20260812 14:18:11.618530 15594 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:11.618706 15594 heartbeater.cc:511] Master 127.8.85.62:42101 requested a full tablet report, sending...
I20260812 14:18:11.619160 15057 ts_manager.cc:194] Registered new tserver with Master: 97f41ff80c1846f78557f95352086b44 (127.8.85.3:35663)
I20260812 14:18:11.619639 15057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:56591
I20260812 14:18:11.621851 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/test-admin.keytab.
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:11.639922 8532 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 14:18:11.641630 15603 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:11.640754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33164 (local address 127.8.85.1:42367)
0812 14:18:11.640924 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:11.640928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:11.640946 (+ 18us) server_negotiation.cc:407] Connection header received
0812 14:18:11.640970 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:11.640972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:11.640985 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:11.641072 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:11.641536 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33164: BlockingRecv error: recv got EOF from 127.0.0.1:33164 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":73,"threads_started":1}
W20260812 14:18:11.641782 8532 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/krb5cc' of type FILE
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:11.642763 8532 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:11.643703 8532 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.1@KRBTEST.COM
I20260812 14:18:11.651865 15603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:11.644203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33168 (local address 127.8.85.1:42367)
0812 14:18:11.644268 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:11.644270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:11.644341 (+ 71us) server_negotiation.cc:407] Connection header received
0812 14:18:11.644451 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:11.644452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:11.644464 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:11.644532 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:11.644960 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.645676 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.646438 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.646755 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.646839 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:11.649011 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:11.649020 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:11.649020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:11.649028 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:11.650601 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:11.650974 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:11.650977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:11.650978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:11.651042 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:11.651255 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:11.651256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:11.651256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:11.651395 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:11.651596 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:11.651669 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 14:18:11.651746 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260812 14:18:11.653584 15603 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:11.653060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33174 (local address 127.8.85.1:42367)
0812 14:18:11.653138 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:11.653140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:11.653149 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:18:11.653210 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:11.653211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:11.653221 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:11.653269 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:11.653537 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33174: BlockingRecv error: recv got EOF from 127.0.0.1:33174 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 14:18:11.653697 8532 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/krb5cc)
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:11.654640 8532 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:11.655609 8532 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.1@KRBTEST.COM
I20260812 14:18:11.662292 15603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:11.656081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33188 (local address 127.8.85.1:42367)
0812 14:18:11.656147 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:11.656149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:11.656185 (+ 36us) server_negotiation.cc:407] Connection header received
0812 14:18:11.656235 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:11.656236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:11.656244 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:11.656288 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:11.656640 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.657099 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.657719 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.657947 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.658008 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:11.660027 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:11.660029 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:11.660029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:11.660035 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:11.661378 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:11.661660 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:11.661661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:11.661661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:11.661709 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:11.661850 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:11.661850 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:11.661850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:11.661947 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:11.662011 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:11.662094 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 14:18:11.662149 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 14:18:11.663717 15603 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:11.663192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33204 (local address 127.8.85.1:42367)
0812 14:18:11.663229 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:11.663236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:11.663272 (+ 36us) server_negotiation.cc:407] Connection header received
0812 14:18:11.663328 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:11.663329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:11.663336 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:11.663382 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:11.663659 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33204: BlockingRecv error: recv got EOF from 127.0.0.1:33204 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 14:18:11.663816 8532 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786544263511571-8532-0/krb5kdc/krb5cc' of type FILE
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:11.664736 8532 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:11.665570 8532 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.1@KRBTEST.COM
I20260812 14:18:11.671970 15603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:11.666000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33214 (local address 127.8.85.1:42367)
0812 14:18:11.666074 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:11.666076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:11.666126 (+ 50us) server_negotiation.cc:407] Connection header received
0812 14:18:11.666179 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:11.666179 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:11.666186 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:11.666229 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:11.666531 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.666977 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.667669 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:11.667836 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:11.667885 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:11.669797 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:11.669798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:11.669799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:11.669804 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:11.670942 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:11.671244 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:11.671245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:11.671245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:11.671291 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:11.671512 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:11.671512 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:11.671513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:11.671595 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:11.671662 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:11.671789 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 14:18:11.671843 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 14:18:11.672632 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 15130
I20260812 14:18:11.681958 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 15291
I20260812 14:18:11.689800 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 15444
I20260812 14:18:11.696580 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 15016
2026-08-12T14:18:11Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1990 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15621](info): set up 2 sockets
Aug 12 14:18:11 dist-test-slave-5k54 krb5kdc[15621](info): commencing operation
krb5kdc: starting...
W20260812 14:18:12.814231 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:18:12 dist-test-slave-5k54 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:18:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:12Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.62.keytab.
Entry for principal oryx/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.62.keytab.
I20260812 14:18:12.942094 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:46175
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.62.keytab
--principal=oryx/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:45063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:46175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:13.023808 15636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:13.024006 15636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:13.024029 15636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:13.025214 15636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:13.025256 15636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:13.025274 15636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:13.025329 15636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:13.025346 15636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:13.026827 15636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:45063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:46175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.62.keytab
--principal=oryx/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:46175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:13.027094 15636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:13.027343 15636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:13.032159 15642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:13.032513 15636 server_base.cc:1034] running on GCE node
W20260812 14:18:13.032241 15641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:13.032166 15644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:13.032868 15636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:13.033169 15636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:13.034349 15636 hybrid_clock.cc:648] HybridClock initialized: now 1786544293034325 us; error 36 us; skew 500 ppm
Aug 12 14:18:13 dist-test-slave-5k54 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544293, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:13.037015 15636 init.cc:377] Logged in from keytab as oryx/127.8.85.62@KRBTEST.COM (short username oryx)
I20260812 14:18:13.037483 15636 webserver.cc:460] Webserver started at http://127.8.85.62:37303/ using document root <none> and password file <none>
I20260812 14:18:13.037725 15636 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:13.037792 15636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:13.037943 15636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:13.039055 15636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "3a7c95c0697f4a4682773716b818d778"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "7e9d683fbba723e5daa2ce616f7aab7b"
server_key_iv: "8130f129dc0f0f4e268ad17478d1db66"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.039413 15636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "3a7c95c0697f4a4682773716b818d778"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "7e9d683fbba723e5daa2ce616f7aab7b"
server_key_iv: "8130f129dc0f0f4e268ad17478d1db66"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.040779 15636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:18:13.041582 15652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:13.041762 15636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:18:13.041862 15636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "3a7c95c0697f4a4682773716b818d778"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "7e9d683fbba723e5daa2ce616f7aab7b"
server_key_iv: "8130f129dc0f0f4e268ad17478d1db66"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.041945 15636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:13.052893 15636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:13.055166 15636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:13.055325 15636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:13.059602 15636 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:46175
I20260812 14:18:13.059631 15716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:46175 every 8 connection(s)
I20260812 14:18:13.059985 15636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:13.060962 15717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:13.063177 15717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778: Bootstrap starting.
I20260812 14:18:13.063817 15717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:13.064073 15717 log.cc:824] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:13.064749 15717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778: No bootstrap required, opened a new log
I20260812 14:18:13.066320 15717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } }
I20260812 14:18:13.066478 15717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:13.066502 15717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a7c95c0697f4a4682773716b818d778, State: Initialized, Role: FOLLOWER
I20260812 14:18:13.066642 15717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } }
I20260812 14:18:13.066745 15717 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:13.066772 15717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:13.066802 15717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:13.066959 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 15636
I20260812 14:18:13.067075 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:13.067261 8532 external_mini_cluster.cc:1428] Setting key 54b74215918d09cff088e44b45508151
I20260812 14:18:13.067937 15717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } }
I20260812 14:18:13.068063 15717 leader_election.cc:302] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a7c95c0697f4a4682773716b818d778; no voters:
I20260812 14:18:13.068239 15717 leader_election.cc:288] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:13.068265 15723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:13.068378 15723 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [term 1 LEADER]: Becoming Leader. State: Replica: 3a7c95c0697f4a4682773716b818d778, State: Running, Role: LEADER
I20260812 14:18:13.068465 15723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } }
I20260812 14:18:13.068526 15717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:13.068765 15723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } }
I20260812 14:18:13.068826 15723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:13.068948 15723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a7c95c0697f4a4682773716b818d778. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } }
I20260812 14:18:13.069006 15723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:13.069141 15727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:13.069700 15727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:13.072664 15727 catalog_manager.cc:1347] Generated new cluster ID: d408c0acde0745cfa8da7d22ad3b1495
I20260812 14:18:13.072721 15727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:18:13 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
I20260812 14:18:13.079175 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.067801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40874 (local address 127.8.85.62:46175)
0812 14:18:13.067995 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:13.068003 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:13.068053 (+ 50us) server_negotiation.cc:407] Connection header received
0812 14:18:13.068189 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:13.068196 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:13.068246 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:13.069230 (+ 984us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:13.069330 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.070741 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.071985 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.072575 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.073003 (+ 428us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.075498 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:13.075502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:13.075502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:13.075514 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:13.077511 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.077899 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.077902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.077903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.078051 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.078253 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.078254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.078254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.078570 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:13.078742 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:13.078810 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 14:18:13.078966 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":102,"threads_started":1}
I20260812 14:18:13.082201 15727 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:13.082870 15727 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:13.090167 15727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778: Generated new TSK 0
I20260812 14:18:13.090341 15727 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.1.keytab.
Entry for principal oryx/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.1.keytab.
I20260812 14:18:13.122676 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:46175
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.1.keytab
--principal=oryx/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:45063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:13.204459 15748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:13.204654 15748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:13.204677 15748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:13.205843 15748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:13.205888 15748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:13.205943 15748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:18:13.207372 15748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:45063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.1.keytab
--principal=oryx/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:46175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:13.207653 15748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:13.207895 15748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:13.212818 15754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:13.212831 15757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:13.212895 15755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:13.213121 15748 server_base.cc:1034] running on GCE node
I20260812 14:18:13.213325 15748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:13.213567 15748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:13.214728 15748 hybrid_clock.cc:648] HybridClock initialized: now 1786544293214711 us; error 49 us; skew 500 ppm
Aug 12 14:18:13 dist-test-slave-5k54 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544293, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:13.217257 15748 init.cc:377] Logged in from keytab as oryx/127.8.85.1@KRBTEST.COM (short username oryx)
I20260812 14:18:13.217679 15748 webserver.cc:460] Webserver started at http://127.8.85.1:43313/ using document root <none> and password file <none>
I20260812 14:18:13.217916 15748 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:13.217988 15748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:13.218137 15748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:13.219185 15748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "1b520e20e3f146f6b9052c6794417e84"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "94a77c197b9b22b0b015572d6d37f64f"
server_key_iv: "1a40b206345ee9059ba34408a06fe94f"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.219591 15748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "1b520e20e3f146f6b9052c6794417e84"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "94a77c197b9b22b0b015572d6d37f64f"
server_key_iv: "1a40b206345ee9059ba34408a06fe94f"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.220912 15748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:18:13.221612 15766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:13.221783 15748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:13.221880 15748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "1b520e20e3f146f6b9052c6794417e84"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "94a77c197b9b22b0b015572d6d37f64f"
server_key_iv: "1a40b206345ee9059ba34408a06fe94f"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.221967 15748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:13.258685 15748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:13.260810 15748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:13.260978 15748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:13.261236 15748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:13.261569 15748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:13.261626 15748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:13.261667 15748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:13.261695 15748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:13.267683 15748 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:39575
I20260812 14:18:13.267745 15897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:39575 every 8 connection(s)
I20260812 14:18:13.268111 15748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:18:13.268518 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 15748
I20260812 14:18:13.268672 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:18:13.268926 8532 external_mini_cluster.cc:1428] Setting key be8d563351b1089a9a3f7d07471ddc65
Aug 12 14:18:13 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544293, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.8.85.2@KRBTEST.COM; defaulting to no policy
I20260812 14:18:13.280329 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.269664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:49199 (local address 127.8.85.62:46175)
0812 14:18:13.269707 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:13.269708 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:13.270050 (+ 342us) server_negotiation.cc:407] Connection header received
0812 14:18:13.270815 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:13.270815 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:13.270828 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:13.270871 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:13.272146 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.272934 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.274479 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.274750 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.274806 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.277393 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:13.277394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:13.277395 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:13.277404 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:13.278883 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.279303 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.279304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.279305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.279367 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.279660 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.279661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.279661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.279782 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:13.279854 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:13.280150 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 14:18:13.280234 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:18:13.280517 15902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.269682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.1:49199)
0812 14:18:13.269886 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:13.269970 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:13.270715 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:13.270954 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:13.270958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:13.271830 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:13.272046 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.272052 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.273045 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.273046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.274390 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.274394 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.274844 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.274845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.275006 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.275620 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:13.275631 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:13.277291 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:13.278980 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.278988 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.278991 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.279223 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.279474 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.279475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.279475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.279589 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.279858 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:13.279865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:13.280010 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 14:18:13.280111 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 14:18:13.280274 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":99,"threads_started":1}
I20260812 14:18:13.281076 15898 heartbeater.cc:347] Connected to a master server at 127.8.85.62:46175
I20260812 14:18:13.281165 15898 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:13.281350 15898 heartbeater.cc:511] Master 127.8.85.62:46175 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.85.2@KRBTEST.COM" created.
I20260812 14:18:13.281987 15674 ts_manager.cc:194] Registered new tserver with Master: 1b520e20e3f146f6b9052c6794417e84 (127.8.85.1:39575)
I20260812 14:18:13.282616 15674 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:49199
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.2.keytab.
Entry for principal oryx/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.2.keytab.
I20260812 14:18:13.299108 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:46175
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.2.keytab
--principal=oryx/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:45063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:13.381453 15906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:13.381644 15906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:13.381667 15906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:13.382803 15906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:13.382846 15906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:13.382915 15906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:18:13.384370 15906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:45063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.2.keytab
--principal=oryx/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:46175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:13.384619 15906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:13.384864 15906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:13.389788 15915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:13.389815 15912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:13.389878 15913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:13.390035 15906 server_base.cc:1034] running on GCE node
I20260812 14:18:13.390236 15906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:13.390486 15906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:13.391644 15906 hybrid_clock.cc:648] HybridClock initialized: now 1786544293391644 us; error 41 us; skew 500 ppm
Aug 12 14:18:13 dist-test-slave-5k54 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544293, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:13.394057 15906 init.cc:377] Logged in from keytab as oryx/127.8.85.2@KRBTEST.COM (short username oryx)
I20260812 14:18:13.394533 15906 webserver.cc:460] Webserver started at http://127.8.85.2:41623/ using document root <none> and password file <none>
I20260812 14:18:13.394759 15906 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:13.394825 15906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:13.394972 15906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:13.396054 15906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "07c6b4a0497142c296ba593b4844ec73"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "fdde3cd16982766c2bf147adefb18f5b"
server_key_iv: "2c61b7c79a1e97833fa9e61f5abb2f6a"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.396425 15906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "07c6b4a0497142c296ba593b4844ec73"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "fdde3cd16982766c2bf147adefb18f5b"
server_key_iv: "2c61b7c79a1e97833fa9e61f5abb2f6a"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.397753 15906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:13.398461 15924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:13.398627 15906 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:13.398725 15906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "07c6b4a0497142c296ba593b4844ec73"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "fdde3cd16982766c2bf147adefb18f5b"
server_key_iv: "2c61b7c79a1e97833fa9e61f5abb2f6a"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.398801 15906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:13.409147 15906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:13.410874 15906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:13.411031 15906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:13.411243 15906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:13.411623 15906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:13.411685 15906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:13.411727 15906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:13.411753 15906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:13.417487 15906 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:33873
I20260812 14:18:13.417536 16060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:33873 every 8 connection(s)
I20260812 14:18:13.417831 15906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:18:13.424788 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 15906
I20260812 14:18:13.424881 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:18:13.425035 8532 external_mini_cluster.cc:1428] Setting key d7f416fb43a85c4601db6d87c59ba571
WARNING: no policy specified for oryx/127.8.85.3@KRBTEST.COM; defaulting to no policy
Aug 12 14:18:13 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544293, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.85.3@KRBTEST.COM" created.
I20260812 14:18:13.437855 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.418548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:33495 (local address 127.8.85.62:46175)
0812 14:18:13.418629 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:13.418631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:13.418878 (+ 247us) server_negotiation.cc:407] Connection header received
0812 14:18:13.419350 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:13.419351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:13.419362 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:13.419482 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:13.420406 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.420902 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.421833 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.422022 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.422073 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.435074 (+ 13001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:13.435076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:13.435076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:13.435087 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:13.436381 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.436780 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.436781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.436781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.436833 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.437132 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.437132 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.437133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.437299 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:13.437373 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:13.437665 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 14:18:13.437758 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:18:13.438057 16063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.418522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.2:33495)
0812 14:18:13.418718 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:13.418769 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:13.419263 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:13.419548 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:13.419551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:13.420131 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:13.420322 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.420328 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.420989 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.420990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.421758 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.421759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.422094 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.422095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.422190 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.433099 (+ 10909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:13.433110 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:13.434791 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:13.436469 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.436476 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.436479 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.436714 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.436914 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.436915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.436916 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.437061 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.437369 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:13.437374 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:13.437517 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 14:18:13.437639 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 14:18:13.437813 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":113,"threads_started":1}
I20260812 14:18:13.438702 16061 heartbeater.cc:347] Connected to a master server at 127.8.85.62:46175
I20260812 14:18:13.438823 16061 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:13.439059 16061 heartbeater.cc:511] Master 127.8.85.62:46175 requested a full tablet report, sending...
I20260812 14:18:13.439549 15674 ts_manager.cc:194] Registered new tserver with Master: 07c6b4a0497142c296ba593b4844ec73 (127.8.85.2:33873)
I20260812 14:18:13.439949 15674 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:33495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.3.keytab.
Entry for principal oryx/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.3.keytab.
I20260812 14:18:13.453680 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:46175
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.3.keytab
--principal=oryx/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:45063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:13.538264 16067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:13.538465 16067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:13.538486 16067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:13.539641 16067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:13.539683 16067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:13.539737 16067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:18:13.541157 16067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:45063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.3.keytab
--principal=oryx/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:46175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:13.541404 16067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:13.541640 16067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:13.546826 16074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:13.546834 16076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:13.546835 16073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:13.546980 16067 server_base.cc:1034] running on GCE node
I20260812 14:18:13.547343 16067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:13.547618 16067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:13.548763 16067 hybrid_clock.cc:648] HybridClock initialized: now 1786544293548754 us; error 27 us; skew 500 ppm
Aug 12 14:18:13 dist-test-slave-5k54 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544293, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:13.551220 16067 init.cc:377] Logged in from keytab as oryx/127.8.85.3@KRBTEST.COM (short username oryx)
I20260812 14:18:13.551719 16067 webserver.cc:460] Webserver started at http://127.8.85.3:46633/ using document root <none> and password file <none>
I20260812 14:18:13.551970 16067 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:13.552040 16067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:13.552189 16067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:13.553213 16067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "0e02dd691fac46a8ab77ceb2c3c27784"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "0e3733f1545b5931831dccbd102e6476"
server_key_iv: "7251698e6f5fdd13fc119ada799507fa"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.553623 16067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "0e02dd691fac46a8ab77ceb2c3c27784"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "0e3733f1545b5931831dccbd102e6476"
server_key_iv: "7251698e6f5fdd13fc119ada799507fa"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.554952 16067 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:18:13.555789 16083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:13.556002 16067 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:13.556104 16067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "0e02dd691fac46a8ab77ceb2c3c27784"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "0e3733f1545b5931831dccbd102e6476"
server_key_iv: "7251698e6f5fdd13fc119ada799507fa"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.556201 16067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:13.573596 16067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:13.575591 16067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:13.575708 16067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:13.575893 16067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:13.576153 16067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:13.576184 16067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:13.576205 16067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:13.576217 16067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:13.582759 16067 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:33573
I20260812 14:18:13.582757 16214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:33573 every 8 connection(s)
I20260812 14:18:13.583220 16067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:18:13.589749 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 16067
I20260812 14:18:13.589856 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:18:13.590106 8532 external_mini_cluster.cc:1428] Setting key 241d19db7e71731ba937e6973a044e5c
Aug 12 14:18:13 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544293, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
I20260812 14:18:13.596688 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.584031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:50687 (local address 127.8.85.62:46175)
0812 14:18:13.584095 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:13.584097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:13.584354 (+ 257us) server_negotiation.cc:407] Connection header received
0812 14:18:13.585144 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:13.585145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:13.585178 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:13.585255 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:13.586650 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.587571 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.589359 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.589680 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.589762 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.593407 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:13.593409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:13.593409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:13.593417 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:13.595135 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.595662 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.595664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.595665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.595732 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.596003 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.596004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.596005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.596142 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:13.596216 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:13.596512 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 14:18:13.596597 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:18:13.596875 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.584047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.3:50687)
0812 14:18:13.584222 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:13.584264 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:13.584948 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:13.585358 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:13.585363 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:13.586227 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:13.586535 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.586541 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.587688 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.587689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.589262 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.589266 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.589856 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.589858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.590018 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.591336 (+ 1318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:13.591345 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:13.593285 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:13.595231 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.595238 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.595240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.595586 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.595802 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.595802 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.595803 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.595934 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.596224 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:13.596230 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:13.596359 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 14:18:13.596484 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 14:18:13.596644 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260812 14:18:13.597312 16216 heartbeater.cc:347] Connected to a master server at 127.8.85.62:46175
I20260812 14:18:13.597430 16216 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:13.597662 16216 heartbeater.cc:511] Master 127.8.85.62:46175 requested a full tablet report, sending...
I20260812 14:18:13.598104 15674 ts_manager.cc:194] Registered new tserver with Master: 0e02dd691fac46a8ab77ceb2c3c27784 (127.8.85.3:33573)
I20260812 14:18:13.598495 15674 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:50687
I20260812 14:18:13.603381 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.85.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.85.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.85.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.61.keytab.
Entry for principal oryx/127.8.85.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.61.keytab.
I20260812 14:18:13.636426 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.61:33609
--webserver_interface=127.8.85.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.61.keytab
--principal=oryx/127.8.85.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:45063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:46175,127.8.85.61:33609
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:13.728649 16222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:13.728866 16222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:13.728891 16222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:13.730080 16222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:13.730119 16222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:13.730135 16222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:13.730149 16222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:13.730162 16222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:13.731863 16222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:45063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:46175,127.8.85.61:33609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.61.keytab
--principal=oryx/127.8.85.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.61:33609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.85.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
W20260812 14:18:13.731973 16222 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.85.62:46175,127.8.85.61:33609'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 14:18:13.732090 16222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:13.732278 16222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:13.737077 16228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:13.737085 16232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:13.737138 16222 server_base.cc:1034] running on GCE node
W20260812 14:18:13.737057 16229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:13.737555 16222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:13.737845 16222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:13.739017 16222 hybrid_clock.cc:648] HybridClock initialized: now 1786544293739002 us; error 27 us; skew 500 ppm
Aug 12 14:18:13 dist-test-slave-5k54 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544293, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:13.741461 16222 init.cc:377] Logged in from keytab as oryx/127.8.85.61@KRBTEST.COM (short username oryx)
I20260812 14:18:13.741830 16222 webserver.cc:460] Webserver started at http://127.8.85.61:33739/ using document root <none> and password file <none>
I20260812 14:18:13.742055 16222 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:13.742113 16222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:13.742205 16222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:13.743242 16222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/data/instance:
uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "6693047284a436f7799981296512d90c"
server_key_iv: "05c882dc48959bfadb84642d9cdcb888"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.743623 16222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/wal/instance:
uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "6693047284a436f7799981296512d90c"
server_key_iv: "05c882dc48959bfadb84642d9cdcb888"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.744900 16222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:18:13.745617 16241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:13.745790 16222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:13.745884 16222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/wal
uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "6693047284a436f7799981296512d90c"
server_key_iv: "05c882dc48959bfadb84642d9cdcb888"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.745939 16222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:13.759718 16222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:13.762036 16222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:13.762143 16222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:13.766372 16222 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.61:33609
I20260812 14:18:13.766446 16301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.61:33609 every 8 connection(s)
I20260812 14:18:13.766784 16222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/data/info.pb
I20260812 14:18:13.767895 16302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:13.769521 16302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:18:13.771984 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 16222
Aug 12 14:18:13 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544293, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.61@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
Aug 12 14:18:13 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.85.61@KRBTEST.COM
I20260812 14:18:13.780112 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.770263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40882 (local address 127.8.85.62:46175)
0812 14:18:13.770323 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:13.770324 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:13.770527 (+ 203us) server_negotiation.cc:407] Connection header received
0812 14:18:13.770948 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:13.770949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:13.770960 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:13.771012 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:13.771627 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.772347 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.773708 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.773896 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.773946 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.776336 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:13.776338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:13.776338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:13.776345 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:13.778540 (+ 2195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.778978 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.778979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.778980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.779046 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.779304 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.779305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.779305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.779535 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:13.779609 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:13.779930 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 14:18:13.780005 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:18:13.780292 16305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.770245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.0.0.1:40882)
0812 14:18:13.770408 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:13.770445 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:13.770867 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:13.771062 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:13.771065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:13.771294 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:13.771556 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.771562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.772515 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.772516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.773624 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.773625 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.773977 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.773978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.774114 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.774642 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:13.774650 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:13.776236 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:13.778629 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.778635 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.778637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.778876 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.779116 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.779116 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.779117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.779231 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.779611 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:13.779618 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:13.779772 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 14:18:13.779891 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 14:18:13.780061 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260812 14:18:13.781036 16302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:18:13.781311 16307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.772590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49004 (local address 127.8.85.61:33609)
0812 14:18:13.772781 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:13.772786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:13.772829 (+ 43us) server_negotiation.cc:407] Connection header received
0812 14:18:13.772873 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:13.772875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:13.772895 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:13.772984 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:13.773526 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.774438 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.775203 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.775543 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.775599 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.778075 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:13.778084 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:13.778084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:13.778093 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:13.779718 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.780092 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.780095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.780096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.780277 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.780703 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.780705 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.780705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.780879 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:13.781020 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:13.781122 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 14:18:13.781200 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":133,"threads_started":1}
Aug 12 14:18:13 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544293, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.61@KRBTEST.COM for oryx/127.8.85.61@KRBTEST.COM
I20260812 14:18:13.787487 16305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.781286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33609 (local address 127.0.0.1:49018)
0812 14:18:13.781453 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:13.781464 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:13.781519 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:13.781665 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:13.781666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:13.781822 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:13.781882 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.781884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.782571 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.782572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.783085 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.783086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.783397 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.783398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.783504 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.783808 (+ 304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:13.783812 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:13.784930 (+ 1118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:13.786505 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.786506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.786506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.786669 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.786866 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.786867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.786867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.786922 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.787175 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:13.787175 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:13.787211 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 14:18:13.787279 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 14:18:13.787374 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260812 14:18:13.787556 16307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.781338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49018 (local address 127.8.85.61:33609)
0812 14:18:13.781464 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:13.781466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:13.781505 (+ 39us) server_negotiation.cc:407] Connection header received
0812 14:18:13.781589 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:13.781590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:13.781599 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:13.781643 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:13.781963 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.782491 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.783154 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.783335 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.783384 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.785034 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:13.785036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:13.785036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:13.785042 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:13.786408 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.786743 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.786744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.786744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.786795 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.787004 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.787004 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.787005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.787107 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:13.787182 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:13.787302 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 14:18:13.787446 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":89,"spinlock_wait_cycles":10624,"thread_start_us":87,"threads_started":1}
I20260812 14:18:13.788750 16302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: Bootstrap starting.
I20260812 14:18:13.789640 16302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:13.789923 16302 log.cc:824] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:13.790706 16302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: No bootstrap required, opened a new log
I20260812 14:18:13.792163 16302 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } }
I20260812 14:18:13.792289 16302 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:13.792311 16302 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf9ce531eb3a4f67bac37f8fed5f05e4, State: Initialized, Role: FOLLOWER
I20260812 14:18:13.792464 16302 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } }
I20260812 14:18:13.792752 16302 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:13.792716 16312 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } } }
I20260812 14:18:13.792819 16312 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:18:13.794520 16326 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:18:13.794597 16326 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:18:13.796118 16222 master_runner.cc:428] Detected that this master cf9ce531eb3a4f67bac37f8fed5f05e4 is joining an existing cluster
I20260812 14:18:13.796203 16222 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.85.61:33609
I20260812 14:18:13.803990 16335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.797229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33609 (local address 127.0.0.1:49024)
0812 14:18:13.797458 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:13.797468 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:13.797537 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:13.797678 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:13.797678 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:13.797863 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:13.797932 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.797933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.798818 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.798819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.799717 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.799720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.800362 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.800363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.800414 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.800823 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:13.800828 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:13.801160 (+ 332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:13.802743 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.802744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.802744 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.803040 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.803261 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.803262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.803262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.803325 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.803634 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:13.803635 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:13.803680 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 14:18:13.803759 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 14:18:13.803868 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260812 14:18:13.804091 16307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.797295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49024 (local address 127.8.85.61:33609)
0812 14:18:13.797403 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:13.797405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:13.797518 (+ 113us) server_negotiation.cc:407] Connection header received
0812 14:18:13.797601 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:13.797602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:13.797608 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:13.797634 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:13.798013 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.798729 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.799813 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.800040 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.800119 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.801271 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:13.801272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:13.801273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:13.801278 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:13.802642 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.803121 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.803122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.803123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.803185 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.803375 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.803376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.803377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.803559 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:13.803631 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:13.803794 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 14:18:13.803940 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":90}
I20260812 14:18:13.804615 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.797406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40892 (local address 127.8.85.62:46175)
0812 14:18:13.797462 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:13.797464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:13.797476 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:18:13.797504 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:13.797505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:13.797516 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:13.797570 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:13.798083 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.798803 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.799573 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.799801 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.799866 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.800955 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:13.800956 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:13.800957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:13.800964 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:13.802726 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.803184 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.803186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.803186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.803251 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.803459 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.803460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.803460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.803585 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:13.803647 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:13.803849 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 14:18:13.803945 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:18:13.805006 16334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.797229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.0.0.1:40892)
0812 14:18:13.797382 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:13.797395 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:13.797454 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:13.797588 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:13.797589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:13.797868 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:13.798000 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.798003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.798886 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.798887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.799484 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.799486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.799871 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.799872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.799945 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.800376 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:13.800381 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:13.800840 (+ 459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:13.802803 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.802804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.802805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.803074 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.803305 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.803306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.803306 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.803376 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.803673 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:13.803674 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:13.803716 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 14:18:13.804824 (+ 1108us) client_negotiation.cc:240] Negotiation successful
0812 14:18:13.804933 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
Aug 12 14:18:13 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544293, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.62@KRBTEST.COM for oryx/127.8.85.61@KRBTEST.COM
I20260812 14:18:13.814983 16307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.807567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49038 (local address 127.8.85.61:33609)
0812 14:18:13.807609 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:13.807611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:13.807825 (+ 214us) server_negotiation.cc:407] Connection header received
0812 14:18:13.807927 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:13.807928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:13.807941 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:13.807987 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:13.808482 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.808917 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.810065 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.810235 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.810286 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.812398 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:13.812400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:13.812400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:13.812407 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:13.813720 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.814136 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.814138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.814138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.814197 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.814424 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.814425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.814426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.814536 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:13.814615 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:13.814800 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 14:18:13.814877 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:18:13.815008 16336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.807554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33609 (local address 127.0.0.1:49038)
0812 14:18:13.807697 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:13.807746 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:13.807849 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:13.807994 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:13.807997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:13.808343 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:13.808406 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.808409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.809007 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.809008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.809991 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.809993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.810308 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.810309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.810391 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.810800 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:13.810806 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:13.812305 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:13.813821 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.813830 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.813832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.814056 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.814267 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.814268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.814269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.814338 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.814608 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:13.814616 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:13.814671 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 14:18:13.814774 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 14:18:13.814894 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260812 14:18:13.815846 15674 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.85.61:33609: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4"
member_type: NON_VOTER
last_known_addr {
host: "127.8.85.61"
port: 33609
}
attrs {
promote: true
}
}
dest_uuid: "3a7c95c0697f4a4682773716b818d778"
cas_config_opid_index: -1
I20260812 14:18:13.816285 15674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: NON_VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: true } }
I20260812 14:18:13.816825 16337 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER cf9ce531eb3a4f67bac37f8fed5f05e4 (127.8.85.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: NON_VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: true } } }
I20260812 14:18:13.817184 16271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:13.818007 16337 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.85.61:33609
I20260812 14:18:13.818113 16271 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 FOLLOWER]: Refusing update from remote peer 3a7c95c0697f4a4682773716b818d778: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 14:18:13.818186 16338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: NON_VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: true } } }
I20260812 14:18:13.818324 16338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:13.818454 16338 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: NON_VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:13.818652 16222 master.cc:537] Master@127.8.85.61:33609 shutting down...
W20260812 14:18:13.819242 15656 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 14:18:13.819705 15656 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 -> Peer cf9ce531eb3a4f67bac37f8fed5f05e4 (127.8.85.61:33609): Couldn't send request to peer cf9ce531eb3a4f67bac37f8fed5f05e4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 14:18:13.821802 16222 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:13.821918 16222 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:13.821966 16222 tablet_replica.cc:330] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: stopping tablet replica
I20260812 14:18:13.825763 16222 master.cc:552] Master@127.8.85.61:33609 shutdown complete.
I20260812 14:18:13.825811 16222 master_runner.cc:305] Clearing existing system tablet
I20260812 14:18:13.825955 16222 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:13.826002 16222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:13.826711 16222 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260812 14:18:13.827252 16342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:13.827392 16222 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:13.827495 16222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:13.827524 16222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/wal
uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "6693047284a436f7799981296512d90c"
server_key_iv: "05c882dc48959bfadb84642d9cdcb888"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.828434 16222 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:13.829896 16222 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 14:18:13.829968 16222 log.cc:1196] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 14:18:13.830164 16222 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: Deleting consensus metadata
I20260812 14:18:13.830287 16222 master_runner.cc:315] Copying system tablet from 127.8.85.62:46175
I20260812 14:18:13.830922 16222 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: tablet copy: Beginning tablet copy session from remote peer at address 127.8.85.62:46175
I20260812 14:18:13.837617 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.831305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40904 (local address 127.8.85.62:46175)
0812 14:18:13.831356 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:13.831358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:13.831479 (+ 121us) server_negotiation.cc:407] Connection header received
0812 14:18:13.831517 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:13.831517 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:13.831534 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:13.831578 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:13.831923 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.832448 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.833591 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.833778 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.833832 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.834931 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:13.834932 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:13.834932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:13.834950 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:13.836439 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.836857 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.836857 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.836858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.836949 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.837175 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.837176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.837176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.837279 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:13.837331 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:13.837492 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 14:18:13.837556 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:18:13.837672 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.831224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.0.0.1:40904)
0812 14:18:13.831354 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:13.831366 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:13.831453 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:13.831583 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:13.831584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:13.831787 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:13.831856 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.831858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.832545 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.832546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.833510 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:13.833513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.833861 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.833862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:13.833932 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.834407 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:13.834412 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:13.834809 (+ 397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:13.836525 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.836526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.836527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.836754 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.837034 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:13.837035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:13.837035 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:13.837105 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:13.837344 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:13.837345 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:13.837393 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 14:18:13.837466 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 14:18:13.837568 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":86,"threads_started":1}
I20260812 14:18:13.838017 15699 tablet_copy_service.cc:140] P 3a7c95c0697f4a4682773716b818d778: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cf9ce531eb3a4f67bac37f8fed5f05e4 ({username='oryx', principal='oryx/127.8.85.61@KRBTEST.COM'} at 127.0.0.1:40904)
I20260812 14:18:13.838117 15699 tablet_copy_service.cc:161] P 3a7c95c0697f4a4682773716b818d778: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cf9ce531eb3a4f67bac37f8fed5f05e4 at {username='oryx', principal='oryx/127.8.85.61@KRBTEST.COM'} at 127.0.0.1:40904: session id = cf9ce531eb3a4f67bac37f8fed5f05e4-00000000000000000000000000000000
I20260812 14:18:13.838894 15699 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 14:18:13.839599 16222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 14:18:13.840555 16222 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: tablet copy: Starting download of 0 data blocks...
I20260812 14:18:13.840718 16222 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: tablet copy: Starting download of 1 WAL segments...
I20260812 14:18:13.841729 16222 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 14:18:13.842525 15699 tablet_copy_service.cc:342] P 3a7c95c0697f4a4682773716b818d778: Request end of tablet copy session cf9ce531eb3a4f67bac37f8fed5f05e4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.85.61@KRBTEST.COM'} at 127.0.0.1:40904
I20260812 14:18:13.842590 15699 tablet_copy_service.cc:433] P 3a7c95c0697f4a4682773716b818d778: ending tablet copy session cf9ce531eb3a4f67bac37f8fed5f05e4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cf9ce531eb3a4f67bac37f8fed5f05e4
I20260812 14:18:13.845293 16222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:13.846498 16352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:13.846549 16351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:13.848326 16354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:13.848443 16222 server_base.cc:1034] running on GCE node
I20260812 14:18:13.848519 16222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:13.848644 16222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:13.849777 16222 hybrid_clock.cc:648] HybridClock initialized: now 1786544293849772 us; error 33 us; skew 500 ppm
Aug 12 14:18:13 dist-test-slave-5k54 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544293, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:13.851084 16222 init.cc:377] Logged in from keytab as oryx/127.8.85.61@KRBTEST.COM (short username oryx)
I20260812 14:18:13.851382 16222 webserver.cc:460] Webserver started at http://127.8.85.61:44387/ using document root <none> and password file <none>
I20260812 14:18:13.851550 16222 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:13.851600 16222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:13.852221 16222 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:13.852739 16362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:13.852862 16222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:13.852906 16222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/wal
uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "6693047284a436f7799981296512d90c"
server_key_iv: "05c882dc48959bfadb84642d9cdcb888"
server_key_version: "encryptionkey@0"
I20260812 14:18:13.852970 16222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:13.864125 16222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:13.865610 16222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:13.869091 16222 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.61:33609
I20260812 14:18:13.869350 16428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.61:33609 every 8 connection(s)
I20260812 14:18:13.869478 16222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-1/data/info.pb
I20260812 14:18:13.870555 16429 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:18:13.871057 16429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: Bootstrap starting.
I20260812 14:18:13.874171 16429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:18:13.874399 16429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: Bootstrap complete.
I20260812 14:18:13.874612 16429 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: NON_VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: true } }
I20260812 14:18:13.874706 16429 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf9ce531eb3a4f67bac37f8fed5f05e4, State: Initialized, Role: LEARNER
I20260812 14:18:13.874747 16429 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: NON_VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: true } }
I20260812 14:18:13.874938 16429 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:13.874930 16432 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: NON_VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: true } } }
I20260812 14:18:13.875059 16432 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: This master's current role is: LEARNER
I20260812 14:18:13.876446 16448 catalog_manager.cc:1259] Loaded cluster ID: d408c0acde0745cfa8da7d22ad3b1495
I20260812 14:18:13.876592 16448 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: loading cluster ID for follower catalog manager: success
I20260812 14:18:13.877573 16448 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: acquiring CA information for follower catalog manager: success
I20260812 14:18:13.877847 16448 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:18:13.889572 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:13.883047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49054 (local address 127.8.85.61:33609)
0812 14:18:13.883165 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:13.883167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:13.883174 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:18:13.883204 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:13.883204 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:13.883213 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:13.883284 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:13.883886 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.884531 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.885264 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:13.885475 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:13.885561 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:13.886808 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:13.886810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:13.886814 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:13.886821 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:13.888395 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.888743 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.888744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.888745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.888834 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:13.889030 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:13.889031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:13.889031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:13.889137 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:13.889242 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:13.889354 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 14:18:13.889434 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
WARNING: no policy specified for oryx/127.8.85.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.85.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.85.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.60.keytab.
Entry for principal oryx/127.8.85.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.60.keytab.
I20260812 14:18:13.918113 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.60:44475
--webserver_interface=127.8.85.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.60.keytab
--principal=oryx/127.8.85.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:45063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:46175,127.8.85.61:33609,127.8.85.60:44475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:14.005015 16453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:14.005257 16453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:14.005295 16453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:14.006458 16453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:14.006527 16453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:14.006551 16453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:14.006573 16453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:14.006603 16453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:14.008178 16453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:45063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:46175,127.8.85.61:33609,127.8.85.60:44475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.60.keytab
--principal=oryx/127.8.85.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.60:44475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.85.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:14.008459 16453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:14.008689 16453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:14.013595 16460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:14.013623 16462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:14.013641 16459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:14.013798 16453 server_base.cc:1034] running on GCE node
I20260812 14:18:14.014057 16453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:14.014341 16453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:14.015486 16453 hybrid_clock.cc:648] HybridClock initialized: now 1786544294015486 us; error 41 us; skew 500 ppm
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:14.018040 16453 init.cc:377] Logged in from keytab as oryx/127.8.85.60@KRBTEST.COM (short username oryx)
I20260812 14:18:14.018500 16453 webserver.cc:460] Webserver started at http://127.8.85.60:38141/ using document root <none> and password file <none>
I20260812 14:18:14.018728 16453 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:14.018815 16453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:14.018977 16453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:14.020117 16453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/data/instance:
uuid: "4f819d82536a40b5ab936550c70cd2d7"
format_stamp: "Formatted at 2026-08-12 14:18:14 on dist-test-slave-5k54"
server_key: "9b2a298e424678cc3763996bd63d8aef"
server_key_iv: "196236491cdbb8424c27f3569ddf6ba9"
server_key_version: "encryptionkey@0"
I20260812 14:18:14.020504 16453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/wal/instance:
uuid: "4f819d82536a40b5ab936550c70cd2d7"
format_stamp: "Formatted at 2026-08-12 14:18:14 on dist-test-slave-5k54"
server_key: "9b2a298e424678cc3763996bd63d8aef"
server_key_iv: "196236491cdbb8424c27f3569ddf6ba9"
server_key_version: "encryptionkey@0"
I20260812 14:18:14.021908 16453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:14.022634 16471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:14.022819 16453 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:14.022946 16453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/wal
uuid: "4f819d82536a40b5ab936550c70cd2d7"
format_stamp: "Formatted at 2026-08-12 14:18:14 on dist-test-slave-5k54"
server_key: "9b2a298e424678cc3763996bd63d8aef"
server_key_iv: "196236491cdbb8424c27f3569ddf6ba9"
server_key_version: "encryptionkey@0"
I20260812 14:18:14.023025 16453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:14.032482 16453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:14.034291 16453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:14.034454 16453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:14.038883 16453 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.60:44475
I20260812 14:18:14.038915 16533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.60:44475 every 8 connection(s)
I20260812 14:18:14.039285 16453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/data/info.pb
I20260812 14:18:14.040360 16534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:14.041988 16534 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:18:14.043919 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 16453
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.60@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.85.60@KRBTEST.COM
I20260812 14:18:14.052088 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.042812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40912 (local address 127.8.85.62:46175)
0812 14:18:14.042879 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.042882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.042936 (+ 54us) server_negotiation.cc:407] Connection header received
0812 14:18:14.043653 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.043654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.043670 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.043749 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.044565 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.045224 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.046332 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.046606 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.046659 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.049064 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.049065 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.049066 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.049076 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.050634 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.051053 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.051055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.051055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.051114 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.051372 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.051373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.051374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.051545 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.051620 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.051895 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.051986 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 14:18:14.052294 16536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.042624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.0.0.1:40912)
0812 14:18:14.042795 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.042863 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.043530 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.043821 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.043825 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.044157 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.044472 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.044478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.045413 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.045415 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.046253 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.046255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.046686 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.046687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.046820 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.047369 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.047381 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.048962 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.050715 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.050722 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.050724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.050965 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.051181 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.051181 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.051182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.051288 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.051610 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.051617 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.051753 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.051863 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.052059 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":75,"threads_started":1}
I20260812 14:18:14.052917 16534 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:18:14.053856 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.044492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35048 (local address 127.8.85.60:44475)
0812 14:18:14.044637 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.044641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.044656 (+ 15us) server_negotiation.cc:407] Connection header received
0812 14:18:14.044676 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.044677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.044704 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.044798 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.045404 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.046425 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.047409 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.047741 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.047797 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.050362 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.050369 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.050370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.050377 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.052100 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.052842 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.052845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.052846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.052896 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.053107 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.053108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.053109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.053312 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.053527 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.053586 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.053647 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.60@KRBTEST.COM for oryx/127.8.85.61@KRBTEST.COM
I20260812 14:18:14.059209 16536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.053163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33609 (local address 127.0.0.1:49060)
0812 14:18:14.053209 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.053221 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.053259 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.053443 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.053443 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.053593 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.053654 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.053656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.054424 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.054425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.055002 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.055004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.055325 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.055326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.055375 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.055751 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.055755 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.056917 (+ 1162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.058265 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.058266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.058266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.058491 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.058653 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.058653 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.058653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.058705 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.058896 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.058897 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.058941 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.059025 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.059124 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 14:18:14.059242 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.053254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49060 (local address 127.8.85.61:33609)
0812 14:18:14.053319 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.053321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.053330 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:18:14.053358 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.053359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.053373 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.053432 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.053737 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.054353 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.055072 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.055262 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.055315 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.056993 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.056995 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.056995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.057001 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.058192 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.058543 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.058544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.058544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.058595 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.058743 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.058743 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.058743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.058827 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.058900 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.059066 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.059146 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:18:14.059866 16534 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.60@KRBTEST.COM for oryx/127.8.85.60@KRBTEST.COM
I20260812 14:18:14.066130 16536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.060094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:44475 (local address 127.0.0.1:35062)
0812 14:18:14.060143 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.060153 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.060197 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.060306 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.060307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.060424 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.060449 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.060450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.061250 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.061251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.061696 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.061697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.062126 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.062127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.062164 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.062481 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.062486 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.063580 (+ 1094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.065154 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.065155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.065155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.065318 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.065525 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.065526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.065526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.065581 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.065819 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.065820 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.065856 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.065923 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.066023 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 14:18:14.066183 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.060126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35062 (local address 127.8.85.60:44475)
0812 14:18:14.060165 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.060167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.060179 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:18:14.060238 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.060239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.060246 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.060303 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.060515 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.061162 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.061776 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.062057 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.062114 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.063660 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.063662 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.063662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.063669 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.065055 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.065401 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.065402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.065402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.065460 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.065651 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.065652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.065652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.065756 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.065827 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.065951 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.066072 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":23}
I20260812 14:18:14.067616 16534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: Bootstrap starting.
I20260812 14:18:14.068538 16534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:14.068900 16534 log.cc:824] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:14.069648 16534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: No bootstrap required, opened a new log
I20260812 14:18:14.071139 16534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } }
I20260812 14:18:14.071271 16534 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:14.071295 16534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f819d82536a40b5ab936550c70cd2d7, State: Initialized, Role: FOLLOWER
I20260812 14:18:14.071481 16534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } }
I20260812 14:18:14.071786 16534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:14.071762 16539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } } }
I20260812 14:18:14.072074 16539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:18:14.073660 16551 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:18:14.073722 16551 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:18:14.077407 16453 master_runner.cc:428] Detected that this master 4f819d82536a40b5ab936550c70cd2d7 is joining an existing cluster
I20260812 14:18:14.077448 16453 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.85.60:44475
I20260812 14:18:14.085870 16561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.078458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.0.0.1:40920)
0812 14:18:14.078607 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.078618 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.078692 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.078905 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.078906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.079235 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.079327 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.079328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.080325 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.080326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.081349 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.081351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.081839 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.081840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.081892 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.082736 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.082740 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.083048 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.084786 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.084787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.084787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.084964 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.085172 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.085173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.085173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.085256 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.085514 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.085515 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.085574 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.085653 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.085756 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":68,"thread_start_us":88,"threads_started":1}
I20260812 14:18:14.085866 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.078496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40920 (local address 127.8.85.62:46175)
0812 14:18:14.078546 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.078549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.078665 (+ 116us) server_negotiation.cc:407] Connection header received
0812 14:18:14.078775 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.078776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.078788 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.078822 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.079448 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.080243 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.081442 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.081737 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.081791 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.083160 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.083162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.083163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.083170 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.084672 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.085047 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.085048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.085049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.085107 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.085330 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.085331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.085332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.085445 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.085521 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.085693 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.085765 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544293, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.62@KRBTEST.COM for oryx/127.8.85.60@KRBTEST.COM
I20260812 14:18:14.103878 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.078714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35070 (local address 127.8.85.60:44475)
0812 14:18:14.078748 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.078750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.079159 (+ 409us) server_negotiation.cc:407] Connection header received
0812 14:18:14.079242 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.079243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.079252 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.079277 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.079748 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.080398 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.081440 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.081709 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.081746 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.100717 (+ 18971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.100719 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.100719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.100726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.102510 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.102938 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.102939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.102940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.103001 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.103275 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.103277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.103277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.103442 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.103488 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.103677 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.103755 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:18:14.104214 16563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.078787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:44475 (local address 127.0.0.1:35070)
0812 14:18:14.079103 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.079113 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.079179 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.079325 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.079325 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.079613 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.079666 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.079668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.080475 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.080475 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.081349 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.081351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.081828 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.081829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.081893 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.099951 (+ 18058us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.099956 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.100415 (+ 459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.102641 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.102642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.102643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.102847 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.103110 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.103111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.103111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.103188 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.103531 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.103531 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.103571 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.103996 (+ 425us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.104097 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":37,"thread_start_us":70,"threads_started":1}
I20260812 14:18:14.105381 16565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.087849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35084 (local address 127.8.85.60:44475)
0812 14:18:14.088003 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.088005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.088012 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:18:14.088051 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.088052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.088060 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.088114 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.088421 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.088883 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.089901 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.090080 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.090126 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.092035 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.092036 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.092037 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.092044 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.103523 (+ 11479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.104130 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.104130 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.104131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.104184 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.104422 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.104423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.104423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.104842 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.104923 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.105198 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.105281 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":111,"threads_started":1}
I20260812 14:18:14.105656 16336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.087813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:44475 (local address 127.0.0.1:35084)
0812 14:18:14.087865 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.087875 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.087911 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.088139 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.088140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.088305 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.088349 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.088351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.088967 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.088968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.089821 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.089824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.090148 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.090150 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.090220 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.090560 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.090565 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.091947 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.103621 (+ 11674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.103622 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.103623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.104038 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.104262 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.104263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.104264 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.104328 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.105031 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.105033 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.105085 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.105173 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.105282 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 14:18:14.106045 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 15748
I20260812 14:18:14.106133 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.078656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49076 (local address 127.8.85.61:33609)
0812 14:18:14.078696 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.078698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.078798 (+ 100us) server_negotiation.cc:407] Connection header received
0812 14:18:14.078917 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.078919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.078932 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.078983 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.079416 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.080211 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.081461 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.081756 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.081799 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.101767 (+ 19968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.101769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.101769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.101778 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.103897 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.105064 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.105065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.105066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.105125 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.105319 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.105320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.105320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.105462 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.105503 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.105931 (+ 428us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.106022 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:18:14.106070 16562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.078467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33609 (local address 127.0.0.1:49076)
0812 14:18:14.078742 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.078751 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.078831 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.078999 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.079000 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.079243 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.079320 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.079321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.080311 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.080311 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.081348 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.081351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.081880 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.081881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.081966 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.100531 (+ 18565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.100536 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.101674 (+ 1138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.104250 (+ 2576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.104251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.104251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.104966 (+ 715us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.105195 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.105196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.105196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.105251 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.105662 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.105663 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.105800 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.105863 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.105964 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":85,"thread_start_us":105,"threads_started":1}
I20260812 14:18:14.116607 15674 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.85.60:44475: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7"
member_type: NON_VOTER
last_known_addr {
host: "127.8.85.60"
port: 44475
}
attrs {
promote: true
}
}
dest_uuid: "3a7c95c0697f4a4682773716b818d778"
cas_config_opid_index: 5
I20260812 14:18:14.116927 15674 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: NON_VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: true } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } }
I20260812 14:18:14.117336 16338 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4f819d82536a40b5ab936550c70cd2d7 (127.8.85.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: NON_VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: true } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } } }
I20260812 14:18:14.118184 16338 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.85.60:44475
I20260812 14:18:14.118273 16338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: NON_VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: true } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } } }
I20260812 14:18:14.118350 16338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:14.118456 16453 master.cc:537] Master@127.8.85.60:44475 shutting down...
W20260812 14:18:14.119805 15657 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 -> Peer 4f819d82536a40b5ab936550c70cd2d7 (127.8.85.60:44475): Couldn't send request to peer 4f819d82536a40b5ab936550c70cd2d7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 14:18:14.121690 16453 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:14.121789 16453 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:14.121837 16453 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: stopping tablet replica
I20260812 14:18:14.122859 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:39575
--local_ip_for_outbound_sockets=127.8.85.1
--tserver_master_addrs=127.8.85.62:46175
--webserver_port=43313
--webserver_interface=127.8.85.1
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.1.keytab
--principal=oryx/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:45063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.85.62:46175,127.8.85.61:33609
--tserver_master_addrs=127.8.85.62:46175,127.8.85.61:33609,127.8.85.60:44475 with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:18:14.123814 16336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.118555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33609 (local address 127.0.0.1:49084)
0812 14:18:14.119088 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.119100 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.119141 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.120306 (+ 1165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.120307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.120310 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:14.120365 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.120367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.121354 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.121356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.122542 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.122546 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.123535 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.123537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.123594 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.123615 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.123671 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.123717 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514}
I20260812 14:18:14.124145 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.119566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49084 (local address 127.8.85.61:33609)
0812 14:18:14.120137 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.120139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.120149 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:18:14.120175 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.120176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.120204 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.120280 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:14.120461 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.121249 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.122888 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.123466 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.123538 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.123622 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.124062 (+ 440us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.124098 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546}
I20260812 14:18:14.124835 16398 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 LEARNER]: Refusing update from remote peer 3a7c95c0697f4a4682773716b818d778: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 14:18:14.125208 16337 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: NON_VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 14:18:14.125790 16398 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4f819d82536a40b5ab936550c70cd2d7 (127.8.85.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: NON_VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: true } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } } }
I20260812 14:18:14.126991 16569 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: NON_VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: true } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } } }
I20260812 14:18:14.127148 16569 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: This master's current role is: LEARNER
I20260812 14:18:14.127213 16337 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778: attempting to promote NON_VOTER cf9ce531eb3a4f67bac37f8fed5f05e4 to VOTER
I20260812 14:18:14.127233 16432 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a7c95c0697f4a4682773716b818d778. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: NON_VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: true } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } } }
I20260812 14:18:14.127313 16432 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: This master's current role is: LEARNER
I20260812 14:18:14.127411 16337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } }
I20260812 14:18:14.128028 16336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.127923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:44475 (local address 127.0.0.1:35094)
0812 14:18:14.127965 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.128001 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:44475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 14:18:14.128244 16398 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 LEARNER]: Refusing update from remote peer 3a7c95c0697f4a4682773716b818d778: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 14:18:14.128379 16336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.128264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:44475 (local address 127.0.0.1:35102)
0812 14:18:14.128331 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.128359 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:44475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 14:18:14.128453 15657 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 -> Peer 4f819d82536a40b5ab936550c70cd2d7 (127.8.85.60:44475): Couldn't send request to peer 4f819d82536a40b5ab936550c70cd2d7. Status: Network error: Client connection negotiation failed: client connection to 127.8.85.60:44475: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 14:18:14.128605 16338 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 14:18:14.129279 16338 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cf9ce531eb3a4f67bac37f8fed5f05e4 (127.8.85.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } } }
I20260812 14:18:14.129428 16398 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cf9ce531eb3a4f67bac37f8fed5f05e4 (127.8.85.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } } }
I20260812 14:18:14.130295 16432 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } } }
I20260812 14:18:14.130367 16432 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:18:14.130422 16338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } } }
I20260812 14:18:14.130517 16338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:14.130589 16338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } } }
I20260812 14:18:14.130653 16338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:14.137250 16453 master.cc:552] Master@127.8.85.60:44475 shutdown complete.
I20260812 14:18:14.137305 16453 master_runner.cc:305] Clearing existing system tablet
I20260812 14:18:14.137450 16453 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:14.137502 16453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:14.138245 16453 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260812 14:18:14.138814 16577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:14.138983 16453 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:14.139046 16453 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:14.139071 16453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/wal
uuid: "4f819d82536a40b5ab936550c70cd2d7"
format_stamp: "Formatted at 2026-08-12 14:18:14 on dist-test-slave-5k54"
server_key: "9b2a298e424678cc3763996bd63d8aef"
server_key_iv: "196236491cdbb8424c27f3569ddf6ba9"
server_key_version: "encryptionkey@0"
I20260812 14:18:14.139889 16453 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:14.141505 16453 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 14:18:14.141579 16453 log.cc:1196] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 14:18:14.141783 16453 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: Deleting consensus metadata
I20260812 14:18:14.141902 16453 master_runner.cc:315] Copying system tablet from 127.8.85.62:46175
I20260812 14:18:14.142535 16453 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: tablet copy: Beginning tablet copy session from remote peer at address 127.8.85.62:46175
I20260812 14:18:14.150259 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.142888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40924 (local address 127.8.85.62:46175)
0812 14:18:14.142933 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.142936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.143072 (+ 136us) server_negotiation.cc:407] Connection header received
0812 14:18:14.143122 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.143123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.143135 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.143186 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.143774 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.144575 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.145725 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.146040 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.146110 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.147364 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.147366 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.147366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.147373 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.149045 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.149444 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.149445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.149445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.149507 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.149739 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.149740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.149740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.149867 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.149934 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.150101 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.150175 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:18:14.150260 16586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.142825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.0.0.1:40924)
0812 14:18:14.142982 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.142994 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.143066 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.143190 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.143191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.143414 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.143682 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.143685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.144682 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.144683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.145627 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.145630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.146120 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.146121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.146193 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.146851 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.146856 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.147263 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.149135 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.149136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.149136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.149365 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.149578 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.149578 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.149579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.149647 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.149937 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.149938 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.149986 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.150060 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.150163 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":65,"threads_started":1}
I20260812 14:18:14.150825 15699 tablet_copy_service.cc:140] P 3a7c95c0697f4a4682773716b818d778: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4f819d82536a40b5ab936550c70cd2d7 ({username='oryx', principal='oryx/127.8.85.60@KRBTEST.COM'} at 127.0.0.1:40924)
I20260812 14:18:14.150882 15699 tablet_copy_service.cc:161] P 3a7c95c0697f4a4682773716b818d778: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4f819d82536a40b5ab936550c70cd2d7 at {username='oryx', principal='oryx/127.8.85.60@KRBTEST.COM'} at 127.0.0.1:40924: session id = 4f819d82536a40b5ab936550c70cd2d7-00000000000000000000000000000000
I20260812 14:18:14.151471 15699 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 14:18:14.151991 16453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 14:18:14.153189 16453 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: tablet copy: Starting download of 0 data blocks...
I20260812 14:18:14.153316 16453 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: tablet copy: Starting download of 1 WAL segments...
I20260812 14:18:14.154192 16453 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 14:18:14.155036 15699 tablet_copy_service.cc:342] P 3a7c95c0697f4a4682773716b818d778: Request end of tablet copy session 4f819d82536a40b5ab936550c70cd2d7-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.85.60@KRBTEST.COM'} at 127.0.0.1:40924
I20260812 14:18:14.155083 15699 tablet_copy_service.cc:433] P 3a7c95c0697f4a4682773716b818d778: ending tablet copy session 4f819d82536a40b5ab936550c70cd2d7-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4f819d82536a40b5ab936550c70cd2d7
I20260812 14:18:14.157665 16453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:14.158838 16587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:14.160233 16588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:14.160612 16453 server_base.cc:1034] running on GCE node
W20260812 14:18:14.168349 16590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:14.168522 16453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:14.168669 16453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:14.169775 16453 hybrid_clock.cc:648] HybridClock initialized: now 1786544294169746 us; error 50 us; skew 500 ppm
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:14.171361 16453 init.cc:377] Logged in from keytab as oryx/127.8.85.60@KRBTEST.COM (short username oryx)
I20260812 14:18:14.171685 16453 webserver.cc:460] Webserver started at http://127.8.85.60:45447/ using document root <none> and password file <none>
I20260812 14:18:14.171813 16453 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:14.171854 16453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:14.172648 16453 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:14.173266 16600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:14.173388 16453 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:14.173429 16453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/wal
uuid: "4f819d82536a40b5ab936550c70cd2d7"
format_stamp: "Formatted at 2026-08-12 14:18:14 on dist-test-slave-5k54"
server_key: "9b2a298e424678cc3763996bd63d8aef"
server_key_iv: "196236491cdbb8424c27f3569ddf6ba9"
server_key_version: "encryptionkey@0"
I20260812 14:18:14.173489 16453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:14.184347 16453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:14.186193 16453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:14.189298 16453 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.60:44475
I20260812 14:18:14.189651 16453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/master-2/data/info.pb
I20260812 14:18:14.191577 16662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.60:44475 every 8 connection(s)
I20260812 14:18:14.192339 16663 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:18:14.192852 16663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: Bootstrap starting.
I20260812 14:18:14.195926 16663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:18:14.196143 16663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: Bootstrap complete.
I20260812 14:18:14.196348 16663 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } }
I20260812 14:18:14.196448 16663 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4f819d82536a40b5ab936550c70cd2d7, State: Initialized, Role: LEARNER
I20260812 14:18:14.196484 16663 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } }
I20260812 14:18:14.196650 16663 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:14.196676 16667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } } }
I20260812 14:18:14.196873 16667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [sys.catalog]: This master's current role is: LEARNER
I20260812 14:18:14.198089 16678 catalog_manager.cc:1259] Loaded cluster ID: d408c0acde0745cfa8da7d22ad3b1495
I20260812 14:18:14.198184 16678 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: loading cluster ID for follower catalog manager: success
I20260812 14:18:14.199355 16678 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: acquiring CA information for follower catalog manager: success
I20260812 14:18:14.199693 16678 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 14:18:14.227145 16566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:14.227365 16566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:14.227402 16566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:14.228569 16566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:14.228638 16566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:14.228684 16566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:18:14.230150 16566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:45063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.1.keytab
--principal=oryx/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:39575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=43313
--tserver_master_addrs=127.8.85.62:46175,127.8.85.61:33609,127.8.85.60:44475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:14.230401 16566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:14.230631 16566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:14.236567 16686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:14.236528 16688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:14.236553 16685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:14.236784 16566 server_base.cc:1034] running on GCE node
I20260812 14:18:14.237056 16566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:14.237299 16566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:14.238449 16566 hybrid_clock.cc:648] HybridClock initialized: now 1786544294238438 us; error 31 us; skew 500 ppm
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:14.240972 16566 init.cc:377] Logged in from keytab as oryx/127.8.85.1@KRBTEST.COM (short username oryx)
I20260812 14:18:14.241482 16566 webserver.cc:460] Webserver started at http://127.8.85.1:43313/ using document root <none> and password file <none>
I20260812 14:18:14.241703 16566 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:14.241787 16566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:14.243142 16566 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:14.243832 16695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:14.244037 16566 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:18:14.244138 16566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "1b520e20e3f146f6b9052c6794417e84"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "94a77c197b9b22b0b015572d6d37f64f"
server_key_iv: "1a40b206345ee9059ba34408a06fe94f"
server_key_version: "encryptionkey@0"
I20260812 14:18:14.244488 16566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:14.261508 16566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:14.263527 16566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:14.263690 16566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:14.263958 16566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:14.264346 16566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:14.264405 16566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:14.264437 16566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:14.264465 16566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:14.271301 16566 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:39575
I20260812 14:18:14.271356 16830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:39575 every 8 connection(s)
I20260812 14:18:14.271764 16566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.60@KRBTEST.COM
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.61@KRBTEST.COM
I20260812 14:18:14.281889 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 16566
I20260812 14:18:14.282061 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 15906
I20260812 14:18:14.287173 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.272507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:46259 (local address 127.8.85.62:46175)
0812 14:18:14.272616 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.272618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.272773 (+ 155us) server_negotiation.cc:407] Connection header received
0812 14:18:14.273905 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.273907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.273921 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.273988 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.274841 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.275484 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.277620 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.277841 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.277908 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.280273 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.280275 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.280275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.280285 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.281849 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.286154 (+ 4305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.286156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.286156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.286212 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.286459 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.286460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.286460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.286589 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.286629 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.286964 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.287044 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 14:18:14.287724 16841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.272906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:55425 (local address 127.8.85.60:44475)
0812 14:18:14.273035 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.273037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.273047 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:18:14.273963 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.273965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.273979 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.274035 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.274861 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.275922 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.277057 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.277371 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.277447 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.279828 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.279830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.279830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.279837 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.284512 (+ 4675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.286302 (+ 1790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.286304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.286305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.286380 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.286631 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.286632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.286633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.286814 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.286892 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.287529 (+ 637us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.287620 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260812 14:18:14.288129 16838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.272442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.1:46259)
0812 14:18:14.272707 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.272718 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.273801 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.274058 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.274081 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.274493 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.274747 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.274753 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.275572 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.275573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.277543 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.277545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.277923 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.277923 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.277985 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.278470 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.278476 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.280176 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.281936 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.281944 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.281946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.286070 (+ 4124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.286276 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.286277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.286277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.286398 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.286668 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.286672 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.286820 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.287302 (+ 482us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.287884 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":1602,"thread_start_us":89,"threads_started":1}
I20260812 14:18:14.288134 16840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.272730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:44475 (local address 127.8.85.1:55425)
0812 14:18:14.272856 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.272867 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.273801 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.274079 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.274080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.274550 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.274748 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.274755 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.276005 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.276006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.276966 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.276968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.277463 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.277464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.277600 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.278154 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.278165 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.279732 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.284590 (+ 4858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.284591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.284592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.286203 (+ 1611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.286467 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.286468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.286469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.286549 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.286923 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.286924 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.286996 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.287094 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.287706 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":31,"thread_start_us":67,"threads_started":1}
I20260812 14:18:14.288897 16833 heartbeater.cc:347] Connected to a master server at 127.8.85.62:46175
I20260812 14:18:14.288972 16831 heartbeater.cc:347] Connected to a master server at 127.8.85.60:44475
I20260812 14:18:14.289031 16831 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:14.289032 16833 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:14.289264 16831 heartbeater.cc:511] Master 127.8.85.60:44475 requested a full tablet report, sending...
I20260812 14:18:14.289273 16833 heartbeater.cc:511] Master 127.8.85.62:46175 requested a full tablet report, sending...
I20260812 14:18:14.289758 15674 ts_manager.cc:194] Re-registered known tserver with Master: 1b520e20e3f146f6b9052c6794417e84 (127.8.85.1:39575)
I20260812 14:18:14.289917 16622 ts_manager.cc:194] Registered new tserver with Master: 1b520e20e3f146f6b9052c6794417e84 (127.8.85.1:39575)
I20260812 14:18:14.290249 15674 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:46259
I20260812 14:18:14.297369 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.272440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33609 (local address 127.8.85.1:44287)
0812 14:18:14.272616 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.272660 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.273812 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.274058 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.274063 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.274544 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.274748 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.274755 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.275739 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.275740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.276966 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.276968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.277804 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.277805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.277863 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.278342 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.278347 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.280805 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.296152 (+ 15347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.296153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.296154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.296452 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.296714 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.296715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.296715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.296787 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.297070 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.297071 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.297105 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.297155 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.297239 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":37,"thread_start_us":97,"threads_started":1}
I20260812 14:18:14.297508 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.272564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:44287 (local address 127.8.85.61:33609)
0812 14:18:14.272655 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.272657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.272744 (+ 87us) server_negotiation.cc:407] Connection header received
0812 14:18:14.273924 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.273925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.273939 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.273972 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.274870 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.275650 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.277057 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.277370 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.277450 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.280896 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.280898 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.280898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.280904 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.288059 (+ 7155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.296536 (+ 8477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.296538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.296538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.296613 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.296836 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.296837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.296838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.297000 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.297077 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.297185 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.297344 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 14:18:14.297987 16835 heartbeater.cc:347] Connected to a master server at 127.8.85.61:33609
I20260812 14:18:14.298040 16835 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:14.298120 16835 heartbeater.cc:511] Master 127.8.85.61:33609 requested a full tablet report, sending...
I20260812 14:18:14.298542 16382 ts_manager.cc:194] Registered new tserver with Master: 1b520e20e3f146f6b9052c6794417e84 (127.8.85.1:39575)
I20260812 14:18:14.300880 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:33873
--local_ip_for_outbound_sockets=127.8.85.2
--tserver_master_addrs=127.8.85.62:46175
--webserver_port=41623
--webserver_interface=127.8.85.2
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.2.keytab
--principal=oryx/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:45063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.85.62:46175,127.8.85.61:33609
--tserver_master_addrs=127.8.85.62:46175,127.8.85.61:33609,127.8.85.60:44475 with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:14.384850 16842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:14.385037 16842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:14.385061 16842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:14.386106 16842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:14.386139 16842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:14.386165 16842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:18:14.387595 16842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:45063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.2.keytab
--principal=oryx/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:33873
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=41623
--tserver_master_addrs=127.8.85.62:46175,127.8.85.61:33609,127.8.85.60:44475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:14.387770 16842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:14.387950 16842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:18:14.392889 16842 server_base.cc:1034] running on GCE node
W20260812 14:18:14.392843 16848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:14.392834 16847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:14.392858 16850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:14.393343 16842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:14.393630 16842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:14.394829 16842 hybrid_clock.cc:648] HybridClock initialized: now 1786544294394788 us; error 27 us; skew 500 ppm
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:14.397486 16842 init.cc:377] Logged in from keytab as oryx/127.8.85.2@KRBTEST.COM (short username oryx)
I20260812 14:18:14.397919 16842 webserver.cc:460] Webserver started at http://127.8.85.2:41623/ using document root <none> and password file <none>
I20260812 14:18:14.398186 16842 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:14.398247 16842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:14.399653 16842 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:18:14.400170 16857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:14.400345 16842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:14.400436 16842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "07c6b4a0497142c296ba593b4844ec73"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "fdde3cd16982766c2bf147adefb18f5b"
server_key_iv: "2c61b7c79a1e97833fa9e61f5abb2f6a"
server_key_version: "encryptionkey@0"
I20260812 14:18:14.400776 16842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:14.411893 16842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:14.413609 16842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:14.413763 16842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:14.413988 16842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:14.414283 16842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:14.414337 16842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:14.414376 16842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:14.414404 16842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:14.420289 16842 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:33873
I20260812 14:18:14.420326 16995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:33873 every 8 connection(s)
I20260812 14:18:14.420688 16842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:18:14.425971 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 16842
I20260812 14:18:14.426128 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 16067
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.60@KRBTEST.COM
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
I20260812 14:18:14.432529 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.421453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:48267 (local address 127.8.85.62:46175)
0812 14:18:14.421507 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.421509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.421737 (+ 228us) server_negotiation.cc:407] Connection header received
0812 14:18:14.422891 (+ 1154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.422892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.422906 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.422939 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.423702 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.424456 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.425652 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.425941 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.426031 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.428952 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.428953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.428954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.428962 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.430692 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.431304 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.431305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.431306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.431368 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.431681 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.431682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.431682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.431822 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.431867 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.432289 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.432381 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:18:14.432636 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.421442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.2:48267)
0812 14:18:14.421603 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.421651 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.422752 (+ 1101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.423009 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.423014 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.423393 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.423591 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.423598 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.424556 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.424557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.425514 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.425516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.426034 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.426035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.426107 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.426834 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.426840 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.428825 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.430798 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.430806 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.430808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.431201 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.431478 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.431479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.431479 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.431592 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.431928 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.431932 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.432102 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.432216 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.432398 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":101,"thread_start_us":82,"threads_started":1}
I20260812 14:18:14.432708 16841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.421913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:33811 (local address 127.8.85.60:44475)
0812 14:18:14.421990 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.421992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.422000 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:18:14.422855 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.422856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.422868 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.422924 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.423693 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.424194 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.425209 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.425430 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.425512 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.428924 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.428926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.428926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.428934 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.430776 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.431280 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.431282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.431282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.431347 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.431682 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.431683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.431683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.431822 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.431867 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.432514 (+ 647us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.432605 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.61@KRBTEST.COM
I20260812 14:18:14.434703 17005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.421714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:44475 (local address 127.8.85.2:33811)
0812 14:18:14.421828 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.421839 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.422759 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.423009 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.423012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.423358 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.423589 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.423594 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.424307 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.424308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.425114 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.425116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.425520 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.425521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.425675 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.426314 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.426323 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.428825 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.430856 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.430856 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.430857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.431201 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.431448 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.431449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.431450 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.431592 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.431923 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.431930 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.432102 (+ 172us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.434460 (+ 2358us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.434567 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":11,"thread_start_us":73,"threads_started":1}
I20260812 14:18:14.435314 16996 heartbeater.cc:347] Connected to a master server at 127.8.85.60:44475
I20260812 14:18:14.435336 16997 heartbeater.cc:347] Connected to a master server at 127.8.85.62:46175
I20260812 14:18:14.435474 16997 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:14.435689 16997 heartbeater.cc:511] Master 127.8.85.62:46175 requested a full tablet report, sending...
I20260812 14:18:14.435474 16996 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:14.435917 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:33573
--local_ip_for_outbound_sockets=127.8.85.3
--tserver_master_addrs=127.8.85.62:46175
--webserver_port=46633
--webserver_interface=127.8.85.3
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.3.keytab
--principal=oryx/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:45063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.85.62:46175,127.8.85.61:33609
--tserver_master_addrs=127.8.85.62:46175,127.8.85.61:33609,127.8.85.60:44475 with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:18:14.435990 16996 heartbeater.cc:511] Master 127.8.85.60:44475 requested a full tablet report, sending...
I20260812 14:18:14.436165 15674 ts_manager.cc:194] Re-registered known tserver with Master: 07c6b4a0497142c296ba593b4844ec73 (127.8.85.2:33873)
I20260812 14:18:14.436295 16622 ts_manager.cc:194] Registered new tserver with Master: 07c6b4a0497142c296ba593b4844ec73 (127.8.85.2:33873)
I20260812 14:18:14.436691 15674 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:48267
I20260812 14:18:14.438006 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.421451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:40587 (local address 127.8.85.61:33609)
0812 14:18:14.421517 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.421519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.421737 (+ 218us) server_negotiation.cc:407] Connection header received
0812 14:18:14.422882 (+ 1145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.422884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.422901 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.422936 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.423702 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.424462 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.426115 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.426413 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.426512 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.434349 (+ 7837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.434351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.434351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.434362 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.436752 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.437132 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.437133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.437133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.437201 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.437417 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.437418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.437418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.437576 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.437647 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.437864 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.437944 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:18:14.438086 17003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.421437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33609 (local address 127.8.85.2:40587)
0812 14:18:14.421603 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.421651 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.422746 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.423011 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.423012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.423389 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.423591 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.423598 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.424556 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.424557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.425514 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.425516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.431941 (+ 6425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.431942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.432024 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.432692 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.432697 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.434231 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.436847 (+ 2616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.436848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.436848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.437066 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.437282 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.437283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.437283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.437349 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.437677 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.437678 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.437725 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.437804 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.437929 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":97,"spinlock_wait_cycles":7040,"thread_start_us":102,"threads_started":1}
I20260812 14:18:14.438464 16999 heartbeater.cc:347] Connected to a master server at 127.8.85.61:33609
I20260812 14:18:14.438512 16999 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:14.438584 16999 heartbeater.cc:511] Master 127.8.85.61:33609 requested a full tablet report, sending...
I20260812 14:18:14.438871 16382 ts_manager.cc:194] Registered new tserver with Master: 07c6b4a0497142c296ba593b4844ec73 (127.8.85.2:33873)
W20260812 14:18:14.521265 17006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:14.521478 17006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:14.521502 17006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:14.522683 17006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:14.522732 17006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:14.522760 17006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:18:14.524281 17006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:45063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.3.keytab
--principal=oryx/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:33573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=46633
--tserver_master_addrs=127.8.85.62:46175,127.8.85.61:33609,127.8.85.60:44475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:14.524549 17006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:14.524808 17006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:14.529978 17016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:14.530037 17014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:14.530018 17011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:14.530299 17006 server_base.cc:1034] running on GCE node
I20260812 14:18:14.530511 17006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:14.530737 17006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:14.531900 17006 hybrid_clock.cc:648] HybridClock initialized: now 1786544294531888 us; error 24 us; skew 500 ppm
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:14.534373 17006 init.cc:377] Logged in from keytab as oryx/127.8.85.3@KRBTEST.COM (short username oryx)
I20260812 14:18:14.534881 17006 webserver.cc:460] Webserver started at http://127.8.85.3:46633/ using document root <none> and password file <none>
I20260812 14:18:14.535125 17006 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:14.535210 17006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:14.536602 17006 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:14.537200 17025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:14.537371 17006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:14.537469 17006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "0e02dd691fac46a8ab77ceb2c3c27784"
format_stamp: "Formatted at 2026-08-12 14:18:13 on dist-test-slave-5k54"
server_key: "0e3733f1545b5931831dccbd102e6476"
server_key_iv: "7251698e6f5fdd13fc119ada799507fa"
server_key_version: "encryptionkey@0"
I20260812 14:18:14.537827 17006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:14.551095 17006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:14.553056 17006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:14.553212 17006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:14.553459 17006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:14.553848 17006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:14.553908 17006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:14.553951 17006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:14.553977 17006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:14.559983 17006 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:33573
I20260812 14:18:14.560032 17157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:33573 every 8 connection(s)
I20260812 14:18:14.560400 17006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:18:14.561744 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 17006
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.61@KRBTEST.COM
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
Aug 12 14:18:14 dist-test-slave-5k54 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544294, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.60@KRBTEST.COM
I20260812 14:18:14.573041 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.561087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:54151 (local address 127.8.85.61:33609)
0812 14:18:14.561131 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.561134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.561650 (+ 516us) server_negotiation.cc:407] Connection header received
0812 14:18:14.563352 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.563354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.563367 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.563418 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.563869 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.564597 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.567043 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.567318 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.567386 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.569686 (+ 2300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.569687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.569687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.569694 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.571404 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.571933 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.571934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.571935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.572005 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.572309 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.572310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.572310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.572451 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.572495 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.572849 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.572932 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:18:14.573589 17165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.561100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33609 (local address 127.8.85.3:54151)
0812 14:18:14.561552 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.561561 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.563244 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.563487 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.563488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.563717 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.563784 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.563786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.564739 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.564740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.566923 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.566925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.567388 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.567389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.567475 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.567882 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.567887 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.569575 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.571528 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.571535 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.571537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.571843 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.572085 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.572086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.572086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.572227 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.572543 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.572549 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.572698 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.573144 (+ 446us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.573337 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":1213,"thread_start_us":97,"threads_started":1}
I20260812 14:18:14.574119 17161 heartbeater.cc:347] Connected to a master server at 127.8.85.61:33609
I20260812 14:18:14.574234 17161 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:14.574452 17161 heartbeater.cc:511] Master 127.8.85.61:33609 requested a full tablet report, sending...
I20260812 14:18:14.575006 16382 ts_manager.cc:194] Registered new tserver with Master: 0e02dd691fac46a8ab77ceb2c3c27784 (127.8.85.3:33573)
I20260812 14:18:14.576596 16841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.561378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:48609 (local address 127.8.85.60:44475)
0812 14:18:14.561483 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.561485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.561497 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:18:14.562849 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.562851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.562864 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.562920 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.563827 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.564607 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.566259 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.566572 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.566613 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.571647 (+ 5034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.571648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.571649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.571655 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.573392 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.575689 (+ 2297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.575690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.575690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.575743 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.575976 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.575977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.575978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.576118 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.576171 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.576373 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.576476 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 14:18:14.576967 17163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.561100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:44475 (local address 127.8.85.3:48609)
0812 14:18:14.561269 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.561304 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.562737 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.562990 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.562994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.563384 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.563712 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.563718 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.564739 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.564740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.566150 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.566153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.566687 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.566687 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.566792 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.567575 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.567585 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.571553 (+ 3968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.575161 (+ 3608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.575161 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.575162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.575602 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.575819 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.575819 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.575819 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.575880 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.576207 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.576209 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.576251 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.576735 (+ 484us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.576848 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":82,"thread_start_us":101,"threads_started":1}
I20260812 14:18:14.577008 17172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.562887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35116 (local address 127.8.85.62:46175)
0812 14:18:14.563047 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.563049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.563665 (+ 616us) server_negotiation.cc:407] Connection header received
0812 14:18:14.563763 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.563764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.563776 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.563821 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.564610 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.565315 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.566371 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.566662 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.566698 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.568868 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.568869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.568870 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.568877 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.575565 (+ 6688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.576016 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.576017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.576017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.576079 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.576316 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.576317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.576318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.576463 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.576545 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.576816 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.576898 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260812 14:18:14.577342 17158 heartbeater.cc:347] Connected to a master server at 127.8.85.60:44475
I20260812 14:18:14.577412 17158 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:14.577486 17158 heartbeater.cc:511] Master 127.8.85.60:44475 requested a full tablet report, sending...
I20260812 14:18:14.577747 16622 ts_manager.cc:194] Registered new tserver with Master: 0e02dd691fac46a8ab77ceb2c3c27784 (127.8.85.3:33573)
I20260812 14:18:14.578333 15722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.561475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:47075 (local address 127.8.85.62:46175)
0812 14:18:14.561537 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.561539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.561774 (+ 235us) server_negotiation.cc:407] Connection header received
0812 14:18:14.563461 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.563462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.563476 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.563512 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.564043 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.564825 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.566239 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.566523 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.566579 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.570597 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.570598 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.570599 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.570607 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.575478 (+ 4871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.576443 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.576462 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.576462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.576533 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.576769 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.576770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.576770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.576901 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.576945 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.578151 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.578239 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 14:18:14.578362 17175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.563280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34462 (local address 127.8.85.60:44475)
0812 14:18:14.563552 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.563554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.565344 (+ 1790us) server_negotiation.cc:407] Connection header received
0812 14:18:14.565371 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.565372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.565384 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.565441 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.568518 (+ 3077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.569323 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.570420 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.570750 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.570814 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.572347 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.572348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.572349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.572356 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.576163 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.576550 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.576551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.576551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.576612 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.577485 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.577487 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.577487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.577673 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.577761 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.578174 (+ 413us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.578259 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":57,"threads_started":1}
I20260812 14:18:14.578712 17164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.561391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.3:47075)
0812 14:18:14.561709 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.561719 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.563348 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.563593 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.563593 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.563857 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:14.563927 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.563930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.564917 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.564918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.566150 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.566153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.566645 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.566646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.566792 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.567620 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:14.567625 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:14.570495 (+ 2870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:14.576138 (+ 5643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.576139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.576140 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.576352 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.576629 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:14.576630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:14.576630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:14.576689 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:14.576993 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:14.576994 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:14.577040 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.577148 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.577260 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":50,"thread_start_us":69,"threads_started":1}
I20260812 14:18:14.578819 17174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.563165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35500 (local address 127.8.85.61:33609)
0812 14:18:14.565090 (+ 1925us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.565092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.565099 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:18:14.565118 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.565119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.565130 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.565189 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.567002 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.567793 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.572450 (+ 4657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.572830 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.572880 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.574287 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.574289 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.574289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.574297 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.576039 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.577689 (+ 1650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.577690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.577690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.577745 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.577942 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.577943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.577943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.578088 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.578172 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.578614 (+ 442us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.578710 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1906,"thread_start_us":58,"threads_started":1}
I20260812 14:18:14.579006 15674 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:18:14.580825 15674 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:18:14.581388 17159 heartbeater.cc:347] Connected to a master server at 127.8.85.62:46175
I20260812 14:18:14.581451 17159 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:14.581523 17159 heartbeater.cc:511] Master 127.8.85.62:46175 requested a full tablet report, sending...
I20260812 14:18:14.581785 15673 ts_manager.cc:194] Re-registered known tserver with Master: 0e02dd691fac46a8ab77ceb2c3c27784 (127.8.85.3:33573)
I20260812 14:18:14.582234 15673 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:47075
I20260812 14:18:14.589144 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.585092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:33573 (local address 127.0.0.1:50768)
0812 14:18:14.585522 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.585532 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.585580 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.585909 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.585910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.585913 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:14.585969 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.585971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.586904 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.586905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.587880 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.587883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.588893 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.588894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.588954 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.588959 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.589021 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.589068 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":5,"thread_start_us":89,"threads_started":1}
I20260812 14:18:14.589205 17186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.585279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50768 (local address 127.8.85.3:33573)
0812 14:18:14.585719 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.585726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.585737 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:18:14.585780 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.585782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.585809 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.585895 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:14.586070 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.586818 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.587976 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.588778 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.588884 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.589062 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.589092 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.589133 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":80,"threads_started":1}
I20260812 14:18:14.589874 17181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.584997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:33873 (local address 127.0.0.1:60394)
0812 14:18:14.585201 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.585214 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.585589 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.585909 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.585910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.585916 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:14.585970 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.585971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.587461 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.587462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.588341 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.588343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.589662 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.589663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.589723 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.589727 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.589780 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.589815 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":40}
I20260812 14:18:14.590070 17081 tablet_service.cc:1467] Processing CreateTablet for tablet 34898f3c33224cd5a0335d6a516f20e7 (DEFAULT_TABLE table=test-table [id=22a90dc3591942bcbc3e6a4077c8a762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:14.590224 17185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.585082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60394 (local address 127.8.85.2:33873)
0812 14:18:14.585724 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.585728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.585738 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:18:14.585780 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.585782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.585800 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.585896 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:14.586072 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.587344 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.588426 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.588992 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.589057 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.589464 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.590036 (+ 572us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.590090 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":98,"threads_started":1}
I20260812 14:18:14.590430 16336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.585049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:39575 (local address 127.0.0.1:58552)
0812 14:18:14.585229 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.585239 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.585281 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.586231 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.586232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.586234 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:14.586268 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.586269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.587638 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.587639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.588843 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.588846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.590181 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.590182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.590250 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.590253 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.590312 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.590358 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":98,"threads_started":1}
I20260812 14:18:14.590447 17187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.585473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58552 (local address 127.8.85.1:39575)
0812 14:18:14.586051 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.586055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.586066 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:18:14.586111 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.586114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.586131 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.586222 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:14.586339 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.587538 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.588959 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.590096 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.590174 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.590251 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.590324 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.590364 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":70,"threads_started":1}
I20260812 14:18:14.590458 17081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34898f3c33224cd5a0335d6a516f20e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:14.590827 16917 tablet_service.cc:1467] Processing CreateTablet for tablet 34898f3c33224cd5a0335d6a516f20e7 (DEFAULT_TABLE table=test-table [id=22a90dc3591942bcbc3e6a4077c8a762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:14.591140 16917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34898f3c33224cd5a0335d6a516f20e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:14.591111 16756 tablet_service.cc:1467] Processing CreateTablet for tablet 34898f3c33224cd5a0335d6a516f20e7 (DEFAULT_TABLE table=test-table [id=22a90dc3591942bcbc3e6a4077c8a762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:14.591378 16756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34898f3c33224cd5a0335d6a516f20e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:14.594141 17188 tablet_bootstrap.cc:492] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784: Bootstrap starting.
I20260812 14:18:14.594655 17189 tablet_bootstrap.cc:492] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73: Bootstrap starting.
I20260812 14:18:14.595083 17188 tablet_bootstrap.cc:654] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:14.595295 17189 tablet_bootstrap.cc:654] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:14.595608 17190 tablet_bootstrap.cc:492] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84: Bootstrap starting.
I20260812 14:18:14.595685 17189 log.cc:824] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:14.595706 17188 log.cc:824] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:14.596312 17190 tablet_bootstrap.cc:654] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:14.596642 17188 tablet_bootstrap.cc:492] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784: No bootstrap required, opened a new log
I20260812 14:18:14.596647 17190 log.cc:824] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:14.596701 17188 ts_tablet_manager.cc:1397] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 14:18:14.597375 17190 tablet_bootstrap.cc:492] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84: No bootstrap required, opened a new log
I20260812 14:18:14.597456 17190 ts_tablet_manager.cc:1397] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:18:14.598868 17190 raft_consensus.cc:348] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } }
I20260812 14:18:14.599020 17190 raft_consensus.cc:374] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:14.598949 17188 raft_consensus.cc:348] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } }
I20260812 14:18:14.599081 17190 raft_consensus.cc:729] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b520e20e3f146f6b9052c6794417e84, State: Initialized, Role: FOLLOWER
I20260812 14:18:14.599124 17188 raft_consensus.cc:374] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:14.599174 17188 raft_consensus.cc:729] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e02dd691fac46a8ab77ceb2c3c27784, State: Initialized, Role: FOLLOWER
I20260812 14:18:14.599213 17190 consensus_queue.cc:260] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } }
I20260812 14:18:14.599313 17188 consensus_queue.cc:260] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } }
I20260812 14:18:14.599515 17190 ts_tablet_manager.cc:1428] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:18:14.599561 16833 heartbeater.cc:503] Master 127.8.85.62:46175 was elected leader, sending a full tablet report...
I20260812 14:18:14.599619 17188 ts_tablet_manager.cc:1428] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 14:18:14.600004 17159 heartbeater.cc:503] Master 127.8.85.62:46175 was elected leader, sending a full tablet report...
I20260812 14:18:14.601655 17189 tablet_bootstrap.cc:492] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73: No bootstrap required, opened a new log
I20260812 14:18:14.601780 17189 ts_tablet_manager.cc:1397] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260812 14:18:14.603256 17189 raft_consensus.cc:348] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } }
I20260812 14:18:14.603371 17189 raft_consensus.cc:374] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:14.603428 17189 raft_consensus.cc:729] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07c6b4a0497142c296ba593b4844ec73, State: Initialized, Role: FOLLOWER
I20260812 14:18:14.603559 17189 consensus_queue.cc:260] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } }
I20260812 14:18:14.603818 17189 ts_tablet_manager.cc:1428] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:18:14.603896 16997 heartbeater.cc:503] Master 127.8.85.62:46175 was elected leader, sending a full tablet report...
I20260812 14:18:14.612839 17175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.610226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34464 (local address 127.8.85.60:44475)
0812 14:18:14.610298 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.610300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.610308 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:18:14.610338 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.610339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.610361 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.610432 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:14.610562 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.611047 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.612081 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.612507 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.612598 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.612671 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.612730 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.612770 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:18:14.612839 16336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.610103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:44475 (local address 127.0.0.1:34464)
0812 14:18:14.610171 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.610183 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.610235 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.610451 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.610452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.610455 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:14.610500 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.610502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.611132 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.611133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.611990 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.611993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.612597 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.612599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.612657 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.612661 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.612724 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.612764 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 14:18:14.613265 16633 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260812 14:18:14.614156 16667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a7c95c0697f4a4682773716b818d778. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } } }
I20260812 14:18:14.614250 16667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [sys.catalog]: This master's current role is: LEARNER
W20260812 14:18:14.671507 17000 tablet.cc:2369] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:18:14.772678 16836 tablet.cc:2369] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:18:14.784523 17197 raft_consensus.cc:482] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:18:14.784646 17197 raft_consensus.cc:504] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } }
I20260812 14:18:14.785033 17197 leader_election.cc:288] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07c6b4a0497142c296ba593b4844ec73 (127.8.85.2:33873), 1b520e20e3f146f6b9052c6794417e84 (127.8.85.1:39575)
I20260812 14:18:14.787923 17187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.785330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:59155 (local address 127.8.85.1:39575)
0812 14:18:14.785373 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.785375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.785383 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:18:14.785412 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.785413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.785432 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.785512 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:14.785687 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.786153 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.787104 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.787560 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.787636 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.787646 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.787799 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.787841 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:18:14.787930 17185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.785277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:42267 (local address 127.8.85.2:33873)
0812 14:18:14.785320 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.785323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.785357 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:18:14.785387 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.785388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.785400 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.785454 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:14.785661 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.786150 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.787151 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.787558 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.787641 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.787652 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.787812 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.787855 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:18:14.787959 17163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.785249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:39575 (local address 127.8.85.3:59155)
0812 14:18:14.785305 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.785316 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.785358 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.785534 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.785534 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.785536 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:14.785594 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.785595 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.786241 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.786242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.787011 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.787013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.787648 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.787649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.787726 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.787728 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.787792 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.787838 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":2}
I20260812 14:18:14.787961 17164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.785194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:33873 (local address 127.8.85.3:42267)
0812 14:18:14.785243 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:14.785257 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:14.785311 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:14.785487 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:14.785488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:14.785490 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:14.785578 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.785580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.786258 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.786258 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.787032 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:14.787034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.787645 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.787646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:14.787720 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.787724 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:14.787802 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 14:18:14.787841 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":44}
I20260812 14:18:14.788331 16779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34898f3c33224cd5a0335d6a516f20e7" candidate_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b520e20e3f146f6b9052c6794417e84" is_pre_election: true
I20260812 14:18:14.788540 16779 raft_consensus.cc:2454] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e02dd691fac46a8ab77ceb2c3c27784 in term 0.
I20260812 14:18:14.788555 16939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34898f3c33224cd5a0335d6a516f20e7" candidate_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07c6b4a0497142c296ba593b4844ec73" is_pre_election: true
I20260812 14:18:14.788727 16939 raft_consensus.cc:2454] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e02dd691fac46a8ab77ceb2c3c27784 in term 0.
I20260812 14:18:14.788796 17028 leader_election.cc:302] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e02dd691fac46a8ab77ceb2c3c27784, 1b520e20e3f146f6b9052c6794417e84; no voters:
I20260812 14:18:14.788995 17197 raft_consensus.cc:2781] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:18:14.789057 17197 raft_consensus.cc:482] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:18:14.789095 17197 raft_consensus.cc:3037] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:14.789832 17197 raft_consensus.cc:504] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } }
I20260812 14:18:14.789990 17197 leader_election.cc:288] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [CANDIDATE]: Term 1 election: Requested vote from peers 07c6b4a0497142c296ba593b4844ec73 (127.8.85.2:33873), 1b520e20e3f146f6b9052c6794417e84 (127.8.85.1:39575)
I20260812 14:18:14.790135 16939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34898f3c33224cd5a0335d6a516f20e7" candidate_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07c6b4a0497142c296ba593b4844ec73"
I20260812 14:18:14.790156 16779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34898f3c33224cd5a0335d6a516f20e7" candidate_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b520e20e3f146f6b9052c6794417e84"
I20260812 14:18:14.790237 16939 raft_consensus.cc:3037] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:14.790234 16779 raft_consensus.cc:3037] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:14.790925 16779 raft_consensus.cc:2454] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e02dd691fac46a8ab77ceb2c3c27784 in term 1.
I20260812 14:18:14.790939 16939 raft_consensus.cc:2454] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e02dd691fac46a8ab77ceb2c3c27784 in term 1.
I20260812 14:18:14.791088 17028 leader_election.cc:302] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e02dd691fac46a8ab77ceb2c3c27784, 1b520e20e3f146f6b9052c6794417e84; no voters:
I20260812 14:18:14.791193 17197 raft_consensus.cc:2781] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:14.791378 17197 raft_consensus.cc:686] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [term 1 LEADER]: Becoming Leader. State: Replica: 0e02dd691fac46a8ab77ceb2c3c27784, State: Running, Role: LEADER
I20260812 14:18:14.791527 17197 consensus_queue.cc:237] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } }
I20260812 14:18:14.792430 15674 catalog_manager.cc:5495] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e02dd691fac46a8ab77ceb2c3c27784 (127.8.85.3). New cstate: current_term: 1 leader_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } health_report { overall_health: HEALTHY } } }
I20260812 14:18:14.793045 16573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: NON_VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: true } } }
I20260812 14:18:14.793154 16573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:14.793969 16337 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778: attempting to promote NON_VOTER 4f819d82536a40b5ab936550c70cd2d7 to VOTER
I20260812 14:18:14.794112 16337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } }
I20260812 14:18:14.794526 16633 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 1 LEARNER]: Refusing update from remote peer 3a7c95c0697f4a4682773716b818d778: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260812 14:18:14.794551 16398 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 FOLLOWER]: Refusing update from remote peer 3a7c95c0697f4a4682773716b818d778: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260812 14:18:14.794785 17207 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 14:18:14.794891 16338 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 14:18:14.795543 17207 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 4f819d82536a40b5ab936550c70cd2d7 (127.8.85.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } } }
I20260812 14:18:14.795717 16398 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 4f819d82536a40b5ab936550c70cd2d7 (127.8.85.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } } }
I20260812 14:18:14.795830 16633 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 4f819d82536a40b5ab936550c70cd2d7 (127.8.85.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } } }
I20260812 14:18:14.796581 17207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } } }
I20260812 14:18:14.796672 17207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:14.796655 16573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } } }
I20260812 14:18:14.796753 16573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:14.796743 17208 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } } }
I20260812 14:18:14.796828 17208 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:18:14.796759 17207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } } }
I20260812 14:18:14.797050 17207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:14.797013 16667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a7c95c0697f4a4682773716b818d778" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } } }
I20260812 14:18:14.797117 16667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:18:14.811126 17162 tablet.cc:2369] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:18:14.813704 17186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.811592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50780 (local address 127.8.85.3:33573)
0812 14:18:14.811652 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.811654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.811671 (+ 17us) server_negotiation.cc:407] Connection header received
0812 14:18:14.811704 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.811705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.811724 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.811785 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:18:14.811900 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.812363 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.813065 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.813249 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.813303 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.813439 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:18:14.813540 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:18:14.813600 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.813632 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.813651 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:18:14.815676 16779 raft_consensus.cc:1261] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84 [term 1 FOLLOWER]: Refusing update from remote peer 0e02dd691fac46a8ab77ceb2c3c27784: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:14.815714 16939 raft_consensus.cc:1261] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73 [term 1 FOLLOWER]: Refusing update from remote peer 0e02dd691fac46a8ab77ceb2c3c27784: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:14.816066 17197 consensus_queue.cc:1035] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:14.816241 17204 consensus_queue.cc:1035] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:14.818475 17210 mvcc.cc:205] Tried to move back new op lower bound from 7317685431561748480 to 7317685431466737664. Current Snapshot: MvccSnapshot[applied={T|T < 7317685431561748480}]
I20260812 14:18:14.819053 17211 mvcc.cc:205] Tried to move back new op lower bound from 7317685431561748480 to 7317685431466737664. Current Snapshot: MvccSnapshot[applied={T|T < 7317685431561748480}]
I20260812 14:18:14.820147 17213 mvcc.cc:205] Tried to move back new op lower bound from 7317685431561748480 to 7317685431466737664. Current Snapshot: MvccSnapshot[applied={T|T < 7317685431561748480}]
I20260812 14:18:14.822321 15674 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:14.822398 15674 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:14.823712 15674 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a7c95c0697f4a4682773716b818d778: Sending DeleteTablet for 3 replicas of tablet 34898f3c33224cd5a0335d6a516f20e7
I20260812 14:18:14.824048 16756 tablet_service.cc:1514] Processing DeleteTablet for tablet 34898f3c33224cd5a0335d6a516f20e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:14 UTC) from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:58552
I20260812 14:18:14.824265 17220 tablet_replica.cc:330] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84: stopping tablet replica
I20260812 14:18:14.824396 17220 raft_consensus.cc:2229] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:14.824373 16917 tablet_service.cc:1514] Processing DeleteTablet for tablet 34898f3c33224cd5a0335d6a516f20e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:14 UTC) from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:60394
I20260812 14:18:14.824542 17220 raft_consensus.cc:2258] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:14.824748 17081 tablet_service.cc:1514] Processing DeleteTablet for tablet 34898f3c33224cd5a0335d6a516f20e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:14 UTC) from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:50768
I20260812 14:18:14.824972 17222 tablet_replica.cc:330] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784: stopping tablet replica
I20260812 14:18:14.825089 17220 ts_tablet_manager.cc:1905] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:14.825101 17222 raft_consensus.cc:2229] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:18:14.825268 17222 raft_consensus.cc:2258] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:14.825739 17222 ts_tablet_manager.cc:1905] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:14.826246 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 15636
I20260812 14:18:14.829164 17220 ts_tablet_manager.cc:1918] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:14.829272 17220 log.cc:1196] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-0/wal/wals/34898f3c33224cd5a0335d6a516f20e7
I20260812 14:18:14.829576 17220 ts_tablet_manager.cc:1939] T 34898f3c33224cd5a0335d6a516f20e7 P 1b520e20e3f146f6b9052c6794417e84: Deleting consensus metadata
I20260812 14:18:14.829875 17222 ts_tablet_manager.cc:1918] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:14.829994 17222 log.cc:1196] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/34898f3c33224cd5a0335d6a516f20e7
I20260812 14:18:14.830271 17222 ts_tablet_manager.cc:1939] T 34898f3c33224cd5a0335d6a516f20e7 P 0e02dd691fac46a8ab77ceb2c3c27784: Deleting consensus metadata
I20260812 14:18:14.831792 17221 tablet_replica.cc:330] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73: stopping tablet replica
I20260812 14:18:14.831964 17221 raft_consensus.cc:2229] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:14.832132 17221 raft_consensus.cc:2258] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:14.832705 17221 ts_tablet_manager.cc:1905] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:14.834365 17221 ts_tablet_manager.cc:1918] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:14.834513 17221 log.cc:1196] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/34898f3c33224cd5a0335d6a516f20e7
I20260812 14:18:14.834839 17221 ts_tablet_manager.cc:1939] T 34898f3c33224cd5a0335d6a516f20e7 P 07c6b4a0497142c296ba593b4844ec73: Deleting consensus metadata
W20260812 14:18:14.837522 17027 connection.cc:535] server connection from 127.0.0.1:50768 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 14:18:14.837668 16696 connection.cc:535] server connection from 127.0.0.1:58552 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 14:18:14.837889 16860 connection.cc:535] server connection from 127.0.0.1:60394 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 14:18:14.847352 17175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.825731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34470 (local address 127.8.85.60:44475)
0812 14:18:14.825770 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.825773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.831632 (+ 5859us) server_negotiation.cc:407] Connection header received
0812 14:18:14.831706 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.831708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.831721 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.831780 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.832885 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.833691 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.834906 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.835246 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.835289 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.844319 (+ 9030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.844320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.844320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.844328 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.846073 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.846487 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.846488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.846489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.846554 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.846790 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.846791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.846792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.846932 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.847033 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.847165 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.847256 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:18:14.847807 17174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.839591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35506 (local address 127.8.85.61:33609)
0812 14:18:14.839636 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.839638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.839690 (+ 52us) server_negotiation.cc:407] Connection header received
0812 14:18:14.839794 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.839795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.839812 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.839844 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.840661 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.841262 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.842333 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.842538 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.842596 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.844185 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.844187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.844187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.844196 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.845961 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.846478 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.846480 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.846480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.846534 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.846834 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.846835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.846836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.847002 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.847109 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.847306 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.847401 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:18:14.854408 16841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:14.839797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34480 (local address 127.8.85.60:44475)
0812 14:18:14.839836 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:14.839838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:14.848159 (+ 8321us) server_negotiation.cc:407] Connection header received
0812 14:18:14.848310 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:14.848311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:14.848323 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:14.848379 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:14.848861 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.849607 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.850259 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:14.850564 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:14.850632 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:14.851776 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:14.851778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:14.851778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:14.851785 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:14.853294 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.853615 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.853617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.853617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.853676 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:14.853862 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:14.853863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:14.853863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:14.853999 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:14.854107 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:14.854230 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 14:18:14.854303 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:18:15.819993 17233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:15.819711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.1:57891)
0812 14:18:15.819860 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:15.819943 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
W20260812 14:18:15.820448 16833 heartbeater.cc:644] Failed to heartbeat to 127.8.85.62:46175 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
I20260812 14:18:15.821399 17234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:15.821147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.2:46833)
0812 14:18:15.821283 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:15.821356 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":60,"threads_started":1}
W20260812 14:18:15.821600 16997 heartbeater.cc:644] Failed to heartbeat to 127.8.85.62:46175 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
I20260812 14:18:15.821674 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:15.821147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.3:54675)
0812 14:18:15.821547 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:15.821619 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":63,"threads_started":1}
W20260812 14:18:15.821883 17159 heartbeater.cc:644] Failed to heartbeat to 127.8.85.62:46175 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
I20260812 14:18:16.324554 17236 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a7c95c0697f4a4682773716b818d778)
I20260812 14:18:16.324703 17236 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } }
I20260812 14:18:16.324954 17236 leader_election.cc:288] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a7c95c0697f4a4682773716b818d778 (127.8.85.62:46175), 4f819d82536a40b5ab936550c70cd2d7 (127.8.85.60:44475)
I20260812 14:18:16.325376 17237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.325124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.0.0.1:35550)
0812 14:18:16.325250 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.325317 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":83,"threads_started":1}
W20260812 14:18:16.325621 16367 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
I20260812 14:18:16.325683 17240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3a7c95c0697f4a4682773716b818d778)
I20260812 14:18:16.325803 17240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } }
I20260812 14:18:16.326031 17237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.325935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.0.0.1:35562)
0812 14:18:16.325979 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.326005 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 14:18:16.326121 16367 leader_election.cc:334] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3a7c95c0697f4a4682773716b818d778 (127.8.85.62:46175): Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
I20260812 14:18:16.326287 17240 leader_election.cc:288] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a7c95c0697f4a4682773716b818d778 (127.8.85.62:46175), cf9ce531eb3a4f67bac37f8fed5f05e4 (127.8.85.61:33609)
I20260812 14:18:16.326357 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.326166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.0.0.1:35576)
0812 14:18:16.326253 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.326299 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73,"thread_start_us":52,"threads_started":1}
W20260812 14:18:16.326661 16605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
I20260812 14:18:16.327148 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.327002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.0.0.1:35590)
0812 14:18:16.327057 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.327119 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 14:18:16.327296 16605 leader_election.cc:334] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3a7c95c0697f4a4682773716b818d778 (127.8.85.62:46175): Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
I20260812 14:18:16.328897 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.325326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:44475 (local address 127.0.0.1:34488)
0812 14:18:16.325418 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.325428 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:16.325611 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:16.325849 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:16.325851 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:16.325854 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:16.325913 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:16.325915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.326906 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.326907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:16.328110 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:16.328113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.328648 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.328649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:16.328713 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:16.328718 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:18:16.328787 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 14:18:16.328832 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":45,"threads_started":1}
I20260812 14:18:16.328912 17241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.325509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34488 (local address 127.8.85.60:44475)
0812 14:18:16.325615 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:16.325616 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:16.325627 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:18:16.325745 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:16.325745 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:16.325758 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:16.325819 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:16.325998 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.326807 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.328203 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.328568 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.328626 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:16.328634 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:16.328812 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 14:18:16.328852 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":60,"threads_started":1}
I20260812 14:18:16.329193 16633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4f819d82536a40b5ab936550c70cd2d7" is_pre_election: true
I20260812 14:18:16.329303 16633 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf9ce531eb3a4f67bac37f8fed5f05e4 in term 1.
I20260812 14:18:16.329452 16367 leader_election.cc:302] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f819d82536a40b5ab936550c70cd2d7, cf9ce531eb3a4f67bac37f8fed5f05e4; no voters: 3a7c95c0697f4a4682773716b818d778
I20260812 14:18:16.329584 17236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 14:18:16.329640 17236 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3a7c95c0697f4a4682773716b818d778)
I20260812 14:18:16.329663 17236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:18:16.329702 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.326420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33609 (local address 127.0.0.1:35510)
0812 14:18:16.326457 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.326467 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:16.326525 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:16.326790 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:16.326790 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:16.326793 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:16.326834 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:16.326835 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.327734 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.327735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:16.328724 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:16.328727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.329453 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.329454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:16.329519 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:16.329522 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:18:16.329596 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 14:18:16.329634 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10,"thread_start_us":41,"threads_started":1}
I20260812 14:18:16.329731 17248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.326488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35510 (local address 127.8.85.61:33609)
0812 14:18:16.326627 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:16.326629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:16.326636 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:18:16.326657 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:16.326658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:16.326668 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:16.326740 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:16.326912 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.327652 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.328805 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.329332 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.329435 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:16.329447 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:16.329601 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 14:18:16.329667 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260812 14:18:16.329983 16398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4f819d82536a40b5ab936550c70cd2d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" is_pre_election: true
I20260812 14:18:16.330451 17236 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } }
I20260812 14:18:16.330587 16398 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f819d82536a40b5ab936550c70cd2d7 in current term 2: Already voted for candidate cf9ce531eb3a4f67bac37f8fed5f05e4 in this term.
I20260812 14:18:16.330598 17236 leader_election.cc:288] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [CANDIDATE]: Term 2 election: Requested vote from peers 3a7c95c0697f4a4682773716b818d778 (127.8.85.62:46175), 4f819d82536a40b5ab936550c70cd2d7 (127.8.85.60:44475)
I20260812 14:18:16.330796 16633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4f819d82536a40b5ab936550c70cd2d7"
I20260812 14:18:16.330843 16604 leader_election.cc:302] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f819d82536a40b5ab936550c70cd2d7; no voters: 3a7c95c0697f4a4682773716b818d778, cf9ce531eb3a4f67bac37f8fed5f05e4
I20260812 14:18:16.330883 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.330672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.0.0.1:35600)
0812 14:18:16.330828 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.330859 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260812 14:18:16.330888 16633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:18:16.331197 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.331096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.0.0.1:35612)
0812 14:18:16.331147 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.331176 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 14:18:16.331280 16367 leader_election.cc:334] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3a7c95c0697f4a4682773716b818d778 (127.8.85.62:46175): Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
I20260812 14:18:16.331748 16633 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf9ce531eb3a4f67bac37f8fed5f05e4 in term 2.
I20260812 14:18:16.331838 17240 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:18:16.331938 16367 leader_election.cc:302] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4f819d82536a40b5ab936550c70cd2d7, cf9ce531eb3a4f67bac37f8fed5f05e4; no voters: 3a7c95c0697f4a4682773716b818d778
I20260812 14:18:16.332068 17236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 14:18:16.332206 17236 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [term 2 LEADER]: Becoming Leader. State: Replica: cf9ce531eb3a4f67bac37f8fed5f05e4, State: Running, Role: LEADER
I20260812 14:18:16.332314 17236 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } }
I20260812 14:18:16.332666 17250 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf9ce531eb3a4f67bac37f8fed5f05e4. Latest consensus state: current_term: 2 leader_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } } }
I20260812 14:18:16.332747 17250 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:16.332931 17252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:16.333438 17252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:16.333662 17252 catalog_manager.cc:1259] Loaded cluster ID: d408c0acde0745cfa8da7d22ad3b1495
I20260812 14:18:16.333699 17252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:18:16.333897 17252 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:16.333981 17252 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: Loaded TSK: 0
I20260812 14:18:16.334079 17252 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:18:16.388489 16382 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:18:16.390137 16382 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:18:16.391155 16633 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [term 2 FOLLOWER]: Refusing update from remote peer cf9ce531eb3a4f67bac37f8fed5f05e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 14:18:16.391193 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.391073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.0.0.1:35644)
0812 14:18:16.391137 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.391166 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 14:18:16.391604 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.391495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.0.0.1:35650)
0812 14:18:16.391542 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.391572 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 14:18:16.391623 17250 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 14:18:16.391779 16367 consensus_peers.cc:487] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 -> Peer 3a7c95c0697f4a4682773716b818d778 (127.8.85.62:46175): Couldn't send request to peer 3a7c95c0697f4a4682773716b818d778. Status: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 14:18:16.392258 17240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf9ce531eb3a4f67bac37f8fed5f05e4. Latest consensus state: current_term: 2 leader_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } } }
I20260812 14:18:16.392383 17240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:18:16.392745 17255 mvcc.cc:205] Tried to move back new op lower bound from 7317685438016008192 to 7317685437777707008. Current Snapshot: MvccSnapshot[applied={T|T < 7317685438016008192}]
I20260812 14:18:16.392849 17236 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } } }
I20260812 14:18:16.392958 17236 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:16.392999 17240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "3a7c95c0697f4a4682773716b818d778" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46175 } } peers { permanent_uuid: "cf9ce531eb3a4f67bac37f8fed5f05e4" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33609 } attrs { promote: false } } peers { permanent_uuid: "4f819d82536a40b5ab936550c70cd2d7" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 44475 } attrs { promote: false } } }
I20260812 14:18:16.393083 17240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f819d82536a40b5ab936550c70cd2d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:18:16.393113 17256 mvcc.cc:205] Tried to move back new op lower bound from 7317685438016008192 to 7317685437777707008. Current Snapshot: MvccSnapshot[applied={T|T < 7317685438016008192 or (T in {7317685438016008192})}]
I20260812 14:18:16.398218 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.394921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:33573 (local address 127.0.0.1:50796)
0812 14:18:16.394967 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.394979 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:16.395048 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:16.395452 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:16.395453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:16.395454 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:16.395496 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:16.395497 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.396385 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.396386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:16.397328 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:16.397331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.398017 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.398018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:16.398081 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:16.398085 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:16.398114 (+ 29us) client_negotiation.cc:240] Negotiation successful
0812 14:18:16.398153 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 14:18:16.398350 17237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.394932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:39575 (local address 127.0.0.1:58556)
0812 14:18:16.395038 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.395049 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:16.395100 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:16.395326 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:16.395327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:16.395331 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:16.395455 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:16.395458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.396208 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.396209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:16.397367 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:16.397370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.398116 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.398117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:16.398190 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:16.398192 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:18:16.398252 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:18:16.398290 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":42}
I20260812 14:18:16.398396 17264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.395066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58556 (local address 127.8.85.1:39575)
0812 14:18:16.395176 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:16.395179 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:16.395188 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:18:16.395216 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:16.395217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:16.395229 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:16.395536 (+ 307us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:16.395583 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.396125 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.397463 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.398027 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.398106 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:16.398120 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:16.398272 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 14:18:16.398322 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":62,"threads_started":1}
I20260812 14:18:16.398489 17265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.395162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50796 (local address 127.8.85.3:33573)
0812 14:18:16.395283 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:16.395285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:16.395293 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:18:16.395322 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:16.395323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:16.395335 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:16.395452 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:16.395549 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.396289 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.397471 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.397923 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.398335 (+ 412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:16.398353 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:16.398372 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 14:18:16.398414 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":73,"threads_started":1}
I20260812 14:18:16.398805 17081 tablet_service.cc:1467] Processing CreateTablet for tablet 4001e14d1ab340ffabe919711a99305d (DEFAULT_TABLE table=test-table [id=7e78ff70368b4e3dbcb48408efa6825c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:16.398815 16756 tablet_service.cc:1467] Processing CreateTablet for tablet 4001e14d1ab340ffabe919711a99305d (DEFAULT_TABLE table=test-table [id=7e78ff70368b4e3dbcb48408efa6825c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:16.398959 17081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4001e14d1ab340ffabe919711a99305d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:16.398962 16756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4001e14d1ab340ffabe919711a99305d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:16.399318 17263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.395025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:33873 (local address 127.0.0.1:60396)
0812 14:18:16.395287 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.395298 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:16.395343 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:16.395676 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:16.395677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:16.395679 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:16.395716 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:16.395717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.396656 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.396657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:16.397845 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:16.397847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.399097 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.399098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:16.399168 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:16.399172 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:16.399221 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:18:16.399255 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":77,"threads_started":1}
I20260812 14:18:16.399904 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.395250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60396 (local address 127.8.85.2:33873)
0812 14:18:16.395522 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:16.395524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:16.395534 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:18:16.395560 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:16.395560 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:16.395573 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:16.395632 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:16.395783 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.396552 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.397941 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.398963 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.399045 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:16.399059 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:16.399472 (+ 413us) server_negotiation.cc:299] Negotiation successful
0812 14:18:16.399517 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":60,"threads_started":1}
I20260812 14:18:16.400293 17267 tablet_bootstrap.cc:492] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84: Bootstrap starting.
I20260812 14:18:16.400332 16917 tablet_service.cc:1467] Processing CreateTablet for tablet 4001e14d1ab340ffabe919711a99305d (DEFAULT_TABLE table=test-table [id=7e78ff70368b4e3dbcb48408efa6825c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:16.400470 16917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4001e14d1ab340ffabe919711a99305d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:16.400697 17269 tablet_bootstrap.cc:492] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784: Bootstrap starting.
I20260812 14:18:16.400833 17267 tablet_bootstrap.cc:654] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:16.401149 17269 tablet_bootstrap.cc:654] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:16.401473 17267 tablet_bootstrap.cc:492] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84: No bootstrap required, opened a new log
I20260812 14:18:16.401589 17267 ts_tablet_manager.cc:1397] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:16.401755 17267 raft_consensus.cc:348] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } }
I20260812 14:18:16.401787 17269 tablet_bootstrap.cc:492] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784: No bootstrap required, opened a new log
I20260812 14:18:16.401834 17267 raft_consensus.cc:374] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:16.401832 17269 ts_tablet_manager.cc:1397] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:16.401865 17267 raft_consensus.cc:729] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b520e20e3f146f6b9052c6794417e84, State: Initialized, Role: FOLLOWER
I20260812 14:18:16.401925 17267 consensus_queue.cc:260] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } }
I20260812 14:18:16.401952 17269 raft_consensus.cc:348] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } }
I20260812 14:18:16.402021 17269 raft_consensus.cc:374] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:16.402045 17269 raft_consensus.cc:729] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e02dd691fac46a8ab77ceb2c3c27784, State: Initialized, Role: FOLLOWER
I20260812 14:18:16.402052 17273 tablet_bootstrap.cc:492] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73: Bootstrap starting.
I20260812 14:18:16.402100 17269 consensus_queue.cc:260] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } }
I20260812 14:18:16.402130 17267 ts_tablet_manager.cc:1428] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:16.402256 17269 ts_tablet_manager.cc:1428] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:16.402514 17273 tablet_bootstrap.cc:654] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:16.402678 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.402438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.3:36449)
0812 14:18:16.402595 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.402640 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260812 14:18:16.403156 17273 tablet_bootstrap.cc:492] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73: No bootstrap required, opened a new log
I20260812 14:18:16.403199 17273 ts_tablet_manager.cc:1397] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:16.403316 17273 raft_consensus.cc:348] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } }
I20260812 14:18:16.403398 17273 raft_consensus.cc:374] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:16.403438 17273 raft_consensus.cc:729] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07c6b4a0497142c296ba593b4844ec73, State: Initialized, Role: FOLLOWER
I20260812 14:18:16.403483 17273 consensus_queue.cc:260] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } }
I20260812 14:18:16.403609 17161 heartbeater.cc:503] Master 127.8.85.61:33609 was elected leader, sending a full tablet report...
I20260812 14:18:16.403646 17273 ts_tablet_manager.cc:1428] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:16.404244 16999 heartbeater.cc:503] Master 127.8.85.61:33609 was elected leader, sending a full tablet report...
I20260812 14:18:16.404600 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.404362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.1:50681)
0812 14:18:16.404497 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.404553 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260812 14:18:16.404976 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.404855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.3:38481)
0812 14:18:16.404913 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.404946 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 14:18:16.405149 16835 heartbeater.cc:503] Master 127.8.85.61:33609 was elected leader, sending a full tablet report...
I20260812 14:18:16.405274 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.403857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.2:59515)
0812 14:18:16.405186 (+ 1329us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.405226 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1289,"thread_start_us":82,"threads_started":1}
I20260812 14:18:16.405416 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.405271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.1:41207)
0812 14:18:16.405363 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.405392 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 14:18:16.590775 17276 raft_consensus.cc:482] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:18:16.590888 17276 raft_consensus.cc:504] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } }
I20260812 14:18:16.591135 17276 leader_election.cc:288] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b520e20e3f146f6b9052c6794417e84 (127.8.85.1:39575), 07c6b4a0497142c296ba593b4844ec73 (127.8.85.2:33873)
I20260812 14:18:16.591413 16779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4001e14d1ab340ffabe919711a99305d" candidate_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b520e20e3f146f6b9052c6794417e84" is_pre_election: true
I20260812 14:18:16.591460 16939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4001e14d1ab340ffabe919711a99305d" candidate_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07c6b4a0497142c296ba593b4844ec73" is_pre_election: true
I20260812 14:18:16.591559 16779 raft_consensus.cc:2454] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e02dd691fac46a8ab77ceb2c3c27784 in term 0.
I20260812 14:18:16.591565 16939 raft_consensus.cc:2454] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e02dd691fac46a8ab77ceb2c3c27784 in term 0.
I20260812 14:18:16.591732 17028 leader_election.cc:302] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e02dd691fac46a8ab77ceb2c3c27784, 1b520e20e3f146f6b9052c6794417e84; no voters:
I20260812 14:18:16.591877 17276 raft_consensus.cc:2781] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:18:16.591934 17276 raft_consensus.cc:482] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:18:16.591950 17276 raft_consensus.cc:3037] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:16.592726 17276 raft_consensus.cc:504] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } }
I20260812 14:18:16.592869 17276 leader_election.cc:288] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [CANDIDATE]: Term 1 election: Requested vote from peers 1b520e20e3f146f6b9052c6794417e84 (127.8.85.1:39575), 07c6b4a0497142c296ba593b4844ec73 (127.8.85.2:33873)
I20260812 14:18:16.593029 16939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4001e14d1ab340ffabe919711a99305d" candidate_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07c6b4a0497142c296ba593b4844ec73"
I20260812 14:18:16.593106 16939 raft_consensus.cc:3037] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:16.593058 16779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4001e14d1ab340ffabe919711a99305d" candidate_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b520e20e3f146f6b9052c6794417e84"
I20260812 14:18:16.593143 16779 raft_consensus.cc:3037] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:16.593727 16779 raft_consensus.cc:2454] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e02dd691fac46a8ab77ceb2c3c27784 in term 1.
I20260812 14:18:16.593788 16939 raft_consensus.cc:2454] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e02dd691fac46a8ab77ceb2c3c27784 in term 1.
I20260812 14:18:16.593959 17028 leader_election.cc:302] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e02dd691fac46a8ab77ceb2c3c27784, 1b520e20e3f146f6b9052c6794417e84; no voters:
I20260812 14:18:16.594059 17276 raft_consensus.cc:2781] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:16.594223 17276 raft_consensus.cc:686] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [term 1 LEADER]: Becoming Leader. State: Replica: 0e02dd691fac46a8ab77ceb2c3c27784, State: Running, Role: LEADER
I20260812 14:18:16.594270 17276 consensus_queue.cc:237] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } } peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } }
I20260812 14:18:16.594726 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.594516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.3:39525)
0812 14:18:16.594640 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.594677 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260812 14:18:16.594975 16381 catalog_manager.cc:5495] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e02dd691fac46a8ab77ceb2c3c27784 (127.8.85.3). New cstate: current_term: 1 leader_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e02dd691fac46a8ab77ceb2c3c27784" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 } health_report { overall_health: UNKNOWN } } }
I20260812 14:18:16.624928 17265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.622604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50806 (local address 127.8.85.3:33573)
0812 14:18:16.622682 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:16.622684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:16.622693 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:18:16.622721 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:16.622722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:16.622741 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:16.622806 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:18:16.622951 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.623485 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.624220 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:16.624419 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:16.624494 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:16.624636 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:18:16.624676 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:18:16.624735 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:16.624805 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 14:18:16.624830 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 14:18:16.625952 16779 raft_consensus.cc:1261] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84 [term 1 FOLLOWER]: Refusing update from remote peer 0e02dd691fac46a8ab77ceb2c3c27784: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:16.625979 16939 raft_consensus.cc:1261] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73 [term 1 FOLLOWER]: Refusing update from remote peer 0e02dd691fac46a8ab77ceb2c3c27784: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:16.626199 17282 consensus_queue.cc:1035] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b520e20e3f146f6b9052c6794417e84" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 39575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:16.626305 17276 consensus_queue.cc:1035] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07c6b4a0497142c296ba593b4844ec73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:16.627470 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.627110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.2:54177)
0812 14:18:16.627361 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.627406 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231}
I20260812 14:18:16.627758 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.627579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.1:46447)
0812 14:18:16.627688 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.627719 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260812 14:18:16.628634 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.628212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.3:52673)
0812 14:18:16.628576 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.628608 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348}
I20260812 14:18:16.628839 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.628655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.2:49597)
0812 14:18:16.628779 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.628810 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260812 14:18:16.628923 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.628844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.3:50305)
0812 14:18:16.628879 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.628899 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 14:18:16.629748 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.629652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46175 (local address 127.8.85.1:49667)
0812 14:18:16.629686 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.629719 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:46175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 14:18:16.630125 16381 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:16.630224 16381 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:16.631496 16381 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf9ce531eb3a4f67bac37f8fed5f05e4: Sending DeleteTablet for 3 replicas of tablet 4001e14d1ab340ffabe919711a99305d
I20260812 14:18:16.631855 16756 tablet_service.cc:1514] Processing DeleteTablet for tablet 4001e14d1ab340ffabe919711a99305d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:16 UTC) from {username='oryx', principal='oryx/127.8.85.61@KRBTEST.COM'} at 127.0.0.1:58556
I20260812 14:18:16.632027 16917 tablet_service.cc:1514] Processing DeleteTablet for tablet 4001e14d1ab340ffabe919711a99305d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:16 UTC) from {username='oryx', principal='oryx/127.8.85.61@KRBTEST.COM'} at 127.0.0.1:60396
I20260812 14:18:16.632079 17294 tablet_replica.cc:330] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84: stopping tablet replica
I20260812 14:18:16.632068 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 16566
I20260812 14:18:16.632135 17294 raft_consensus.cc:2229] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:16.632181 17294 raft_consensus.cc:2258] T 4001e14d1ab340ffabe919711a99305d P 1b520e20e3f146f6b9052c6794417e84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:16.633473 17295 tablet_replica.cc:330] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73: stopping tablet replica
I20260812 14:18:16.633527 17295 raft_consensus.cc:2229] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:16.633567 17295 raft_consensus.cc:2258] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:16.636019 17081 tablet_service.cc:1514] Processing DeleteTablet for tablet 4001e14d1ab340ffabe919711a99305d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:16 UTC) from {username='oryx', principal='oryx/127.8.85.61@KRBTEST.COM'} at 127.0.0.1:50796
I20260812 14:18:16.636344 17296 tablet_replica.cc:330] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784: stopping tablet replica
I20260812 14:18:16.636482 17296 raft_consensus.cc:2229] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:18:16.636569 17295 ts_tablet_manager.cc:1905] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:16.636596 17296 raft_consensus.cc:2258] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:16.637305 17296 ts_tablet_manager.cc:1905] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:16.638293 17295 ts_tablet_manager.cc:1918] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:16.638370 17295 log.cc:1196] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/4001e14d1ab340ffabe919711a99305d
I20260812 14:18:16.638604 17295 ts_tablet_manager.cc:1939] T 4001e14d1ab340ffabe919711a99305d P 07c6b4a0497142c296ba593b4844ec73: Deleting consensus metadata
I20260812 14:18:16.638931 16367 catalog_manager.cc:4841] TS 07c6b4a0497142c296ba593b4844ec73 (127.8.85.2:33873): tablet 4001e14d1ab340ffabe919711a99305d (table test-table [id=7e78ff70368b4e3dbcb48408efa6825c]) successfully deleted
I20260812 14:18:16.639334 17296 ts_tablet_manager.cc:1918] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:16.639410 17296 log.cc:1196] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/4001e14d1ab340ffabe919711a99305d
I20260812 14:18:16.639714 17296 ts_tablet_manager.cc:1939] T 4001e14d1ab340ffabe919711a99305d P 0e02dd691fac46a8ab77ceb2c3c27784: Deleting consensus metadata
I20260812 14:18:16.640090 16367 catalog_manager.cc:4841] TS 0e02dd691fac46a8ab77ceb2c3c27784 (127.8.85.3:33573): tablet 4001e14d1ab340ffabe919711a99305d (table test-table [id=7e78ff70368b4e3dbcb48408efa6825c]) successfully deleted
I20260812 14:18:16.642370 17263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:16.642246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:39575 (local address 127.0.0.1:58562)
0812 14:18:16.642292 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:16.642324 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:39575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 14:18:16.642547 16365 catalog_manager.cc:4584] TS 1b520e20e3f146f6b9052c6794417e84 (127.8.85.1:39575): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4001e14d1ab340ffabe919711a99305d: Network error: Client connection negotiation failed: client connection to 127.8.85.1:39575: connect: Connection refused (error 111)
I20260812 14:18:16.642576 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 16842
I20260812 14:18:16.648380 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 17006
I20260812 14:18:16.656314 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 16222
I20260812 14:18:16.664438 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 16453
2026-08-12T14:18:16Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4970 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:18:16 dist-test-slave-5k54 krb5kdc[17300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:18:16 dist-test-slave-5k54 krb5kdc[17300](info): set up 2 sockets
Aug 12 14:18:16 dist-test-slave-5k54 krb5kdc[17300](info): commencing operation
krb5kdc: starting...
W20260812 14:18:17.838569 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:18:17 dist-test-slave-5k54 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:18:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.62.keytab.
Entry for principal oryx/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.62.keytab.
I20260812 14:18:17.968097 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:33823
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.62.keytab
--principal=oryx/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:33823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:18.049883 17315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:18.050125 17315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:18.050166 17315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:18.051415 17315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:18.051501 17315 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 14:18:18.051522 17315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:18.051544 17315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:18.051574 17315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:18.051596 17315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:18.053092 17315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:33823
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.62.keytab
--principal=oryx/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:33823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:18.053390 17315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:18.053615 17315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:18.058476 17323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:18.058490 17321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:18.058498 17320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:18.058584 17315 server_base.cc:1034] running on GCE node
I20260812 14:18:18.059059 17315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:18.059387 17315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:18.060586 17315 hybrid_clock.cc:648] HybridClock initialized: now 1786544298060578 us; error 32 us; skew 500 ppm
Aug 12 14:18:18 dist-test-slave-5k54 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544298, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:18.063067 17315 init.cc:377] Logged in from keytab as oryx/127.8.85.62@KRBTEST.COM (short username oryx)
I20260812 14:18:18.063567 17315 webserver.cc:460] Webserver started at http://127.8.85.62:44157/ using document root <none> and password file <none>
I20260812 14:18:18.063795 17315 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:18.063862 17315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:18.064002 17315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:18.065061 17315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "d9ccb7b96b5a4438ae0bd40d6962e215"
format_stamp: "Formatted at 2026-08-12 14:18:18 on dist-test-slave-5k54"
server_key: "78d41cf84de07ec59b4d75cff518ecda"
server_key_iv: "c2a1d34f1939aee420277fe8fa47fc26"
server_key_version: "encryptionkey@0"
I20260812 14:18:18.065431 17315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "d9ccb7b96b5a4438ae0bd40d6962e215"
format_stamp: "Formatted at 2026-08-12 14:18:18 on dist-test-slave-5k54"
server_key: "78d41cf84de07ec59b4d75cff518ecda"
server_key_iv: "c2a1d34f1939aee420277fe8fa47fc26"
server_key_version: "encryptionkey@0"
I20260812 14:18:18.066784 17315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:18:18.067612 17332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:18.067790 17315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:18.067896 17315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "d9ccb7b96b5a4438ae0bd40d6962e215"
format_stamp: "Formatted at 2026-08-12 14:18:18 on dist-test-slave-5k54"
server_key: "78d41cf84de07ec59b4d75cff518ecda"
server_key_iv: "c2a1d34f1939aee420277fe8fa47fc26"
server_key_version: "encryptionkey@0"
I20260812 14:18:18.067971 17315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:18.084817 17315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:18.086689 17315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:18.086858 17315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:18.091831 17315 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:33823
I20260812 14:18:18.091882 17406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:33823 every 8 connection(s)
I20260812 14:18:18.092222 17315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:18.092756 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 17315
I20260812 14:18:18.092861 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:18.092983 17407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:18.093125 8532 external_mini_cluster.cc:1428] Setting key 52fe36d267ca54efb1675fe5df32c6f0
I20260812 14:18:18.096664 17407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215: Bootstrap starting.
I20260812 14:18:18.097415 17407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:18.097734 17407 log.cc:824] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:18.098443 17407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215: No bootstrap required, opened a new log
Aug 12 14:18:18 dist-test-slave-5k54 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
I20260812 14:18:18.101053 17407 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ccb7b96b5a4438ae0bd40d6962e215" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33823 } }
I20260812 14:18:18.101217 17407 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:18.101265 17407 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9ccb7b96b5a4438ae0bd40d6962e215, State: Initialized, Role: FOLLOWER
I20260812 14:18:18.101449 17407 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ccb7b96b5a4438ae0bd40d6962e215" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33823 } }
I20260812 14:18:18.101548 17407 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:18.101595 17407 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:18.101642 17407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:18.102150 17407 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ccb7b96b5a4438ae0bd40d6962e215" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33823 } }
I20260812 14:18:18.102236 17407 leader_election.cc:302] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9ccb7b96b5a4438ae0bd40d6962e215; no voters:
I20260812 14:18:18.102451 17407 leader_election.cc:288] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:18.102481 17414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:18.102591 17414 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [term 1 LEADER]: Becoming Leader. State: Replica: d9ccb7b96b5a4438ae0bd40d6962e215, State: Running, Role: LEADER
I20260812 14:18:18.102692 17414 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ccb7b96b5a4438ae0bd40d6962e215" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33823 } }
I20260812 14:18:18.102725 17407 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:18.103021 17414 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9ccb7b96b5a4438ae0bd40d6962e215" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ccb7b96b5a4438ae0bd40d6962e215" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33823 } } }
I20260812 14:18:18.103128 17414 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:18.103286 17414 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9ccb7b96b5a4438ae0bd40d6962e215. Latest consensus state: current_term: 1 leader_uuid: "d9ccb7b96b5a4438ae0bd40d6962e215" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ccb7b96b5a4438ae0bd40d6962e215" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33823 } } }
I20260812 14:18:18.103386 17420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:18.103447 17414 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:18.104125 17420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:18.105294 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.093653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52238 (local address 127.8.85.62:33823)
0812 14:18:18.093845 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.093856 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.093889 (+ 33us) server_negotiation.cc:407] Connection header received
0812 14:18:18.094612 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.094623 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.094700 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.094858 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:18.095645 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.097036 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.098026 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.098443 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.098563 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.101215 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:18.101221 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:18.101221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:18.101237 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:18.103351 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.103811 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.103814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.103814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.103922 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.104150 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.104151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.104151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.104639 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:18.104816 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.104949 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.105091 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260812 14:18:18.106055 17420 catalog_manager.cc:1347] Generated new cluster ID: 4421c7070bc44883b19b3eed7c79679a
I20260812 14:18:18.106105 17420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:18:18.121691 17420 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:18.122251 17420 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:18.127192 17420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215: Generated new TSK 0
I20260812 14:18:18.127403 17420 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.1.keytab.
Entry for principal oryx/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.1.keytab.
I20260812 14:18:18.161916 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33823
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.1.keytab
--principal=oryx/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:18.243983 17436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:18.244215 17436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:18.244254 17436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:18.245412 17436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:18.245479 17436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:18.245518 17436 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:18:18.245561 17436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:18:18.247057 17436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.1.keytab
--principal=oryx/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.85.62:33823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:18.247335 17436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:18.247606 17436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:18.252533 17442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:18.252532 17443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:18.252617 17445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:18.253005 17436 server_base.cc:1034] running on GCE node
I20260812 14:18:18.253211 17436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:18.253408 17436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:18.254554 17436 hybrid_clock.cc:648] HybridClock initialized: now 1786544298254542 us; error 29 us; skew 500 ppm
Aug 12 14:18:18 dist-test-slave-5k54 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544298, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:18.257215 17436 init.cc:377] Logged in from keytab as oryx/127.8.85.1@KRBTEST.COM (short username oryx)
I20260812 14:18:18.257738 17436 webserver.cc:460] Webserver started at http://127.8.85.1:36833/ using document root <none> and password file <none>
I20260812 14:18:18.258000 17436 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:18.258080 17436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:18.258244 17436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:18.259483 17436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "32284027752745229022fbd48077dcbb"
format_stamp: "Formatted at 2026-08-12 14:18:18 on dist-test-slave-5k54"
server_key: "b4bc6bffcddb7fec3184bfb395d563e6"
server_key_iv: "46c53cbef268f360747bf5ab93d6e8ed"
server_key_version: "encryptionkey@0"
I20260812 14:18:18.259907 17436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "32284027752745229022fbd48077dcbb"
format_stamp: "Formatted at 2026-08-12 14:18:18 on dist-test-slave-5k54"
server_key: "b4bc6bffcddb7fec3184bfb395d563e6"
server_key_iv: "46c53cbef268f360747bf5ab93d6e8ed"
server_key_version: "encryptionkey@0"
I20260812 14:18:18.261348 17436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:18.262102 17453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:18.262276 17436 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:18.262408 17436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "32284027752745229022fbd48077dcbb"
format_stamp: "Formatted at 2026-08-12 14:18:18 on dist-test-slave-5k54"
server_key: "b4bc6bffcddb7fec3184bfb395d563e6"
server_key_iv: "46c53cbef268f360747bf5ab93d6e8ed"
server_key_version: "encryptionkey@0"
I20260812 14:18:18.262498 17436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:18.272389 17436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:18.274859 17436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:18.275038 17436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:18.275703 17436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:18.275782 17436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:18.275817 17436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:18.275839 17436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 14:18:18 dist-test-slave-5k54 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544298, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
I20260812 14:18:18.284029 17436 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:42667
I20260812 14:18:18.284099 17592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:42667 every 8 connection(s)
I20260812 14:18:18.284560 17436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:18:18.285768 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.276076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:42755 (local address 127.8.85.62:33823)
0812 14:18:18.276118 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.276120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.276328 (+ 208us) server_negotiation.cc:407] Connection header received
0812 14:18:18.276844 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.276845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.276858 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.276918 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:18.277505 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.278015 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.279032 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.279256 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.279345 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.282494 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:18.282496 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:18.282496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:18.282504 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:18.284414 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.284878 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.284880 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.284880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.284943 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.285220 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.285221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.285221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.285349 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:18.285415 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.285632 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.285702 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:18:18.285961 17465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.276070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33823 (local address 127.8.85.1:42755)
0812 14:18:18.276217 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:18.276250 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:18.276740 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:18.276994 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:18.277001 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:18.277228 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:18.277406 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.277409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.278121 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.278122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.278929 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.278932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.279358 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.279359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.279537 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.280026 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:18.280038 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:18.282389 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:18.284509 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:18.284514 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:18.284516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:18.284799 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:18.285024 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:18.285025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:18.285025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:18.285155 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:18.285416 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:18.285421 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:18.285538 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 14:18:18.285618 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 14:18:18.285733 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
I20260812 14:18:18.286525 17593 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33823
I20260812 14:18:18.286641 17593 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:18.286882 17593 heartbeater.cc:511] Master 127.8.85.62:33823 requested a full tablet report, sending...
I20260812 14:18:18.287252 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 17436
I20260812 14:18:18.287379 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:18:18.287564 17348 ts_manager.cc:194] Registered new tserver with Master: 32284027752745229022fbd48077dcbb (127.8.85.1:42667)
I20260812 14:18:18.287663 8532 external_mini_cluster.cc:1428] Setting key 9e9641d5e7f155c61bae9599bfff49cc
I20260812 14:18:18.288516 17348 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:42755
I20260812 14:18:18.294174 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.287861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:41457 (local address 127.8.85.62:33823)
0812 14:18:18.287900 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.287902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.287909 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:18:18.287929 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.287930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.287946 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.287994 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:18.288314 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.289011 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.289898 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.290212 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.290284 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.291599 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:18.291600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:18.291600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:18.291607 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:18.293010 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.293373 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.293374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.293374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.293447 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.293657 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.293658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.293658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.293770 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:18.293845 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.294011 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.294087 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:18:18.294188 17599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.287598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33823 (local address 127.8.85.1:41457)
0812 14:18:18.287750 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:18.287763 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:18.287817 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:18.288012 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:18.288013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:18.288181 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:18.288228 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.288229 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.289117 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.289117 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.289810 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.289812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.290344 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.290346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.290419 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.291156 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:18.291160 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:18.291510 (+ 350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:18.293118 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:18.293119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:18.293120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:18.293299 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:18.293522 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:18.293522 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:18.293523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:18.293589 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:18.293837 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:18.293838 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:18.293882 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 14:18:18.293982 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 14:18:18.294085 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for oryx/127.8.85.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.2.keytab.
Entry for principal oryx/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.2.keytab.
I20260812 14:18:18.322099 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33823
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.2.keytab
--principal=oryx/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:18.404138 17602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:18.404336 17602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:18.404361 17602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:18.405481 17602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:18.405519 17602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:18.405539 17602 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:18:18.405556 17602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:18:18.406970 17602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.2.keytab
--principal=oryx/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.85.62:33823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:18.407176 17602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:18.407354 17602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:18.412312 17608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:18.412331 17612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:18.412429 17602 server_base.cc:1034] running on GCE node
W20260812 14:18:18.412304 17607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:18.412654 17602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:18.412925 17602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:18.414083 17602 hybrid_clock.cc:648] HybridClock initialized: now 1786544298414064 us; error 30 us; skew 500 ppm
Aug 12 14:18:18 dist-test-slave-5k54 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544298, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:18.416893 17602 init.cc:377] Logged in from keytab as oryx/127.8.85.2@KRBTEST.COM (short username oryx)
I20260812 14:18:18.417359 17602 webserver.cc:460] Webserver started at http://127.8.85.2:33289/ using document root <none> and password file <none>
I20260812 14:18:18.417562 17602 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:18.417603 17602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:18.417693 17602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:18.418705 17602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "aa0b5916c8ad44a191e11342b26c076d"
format_stamp: "Formatted at 2026-08-12 14:18:18 on dist-test-slave-5k54"
server_key: "feb2f83786aff83d7d0d6d240eb2f18a"
server_key_iv: "085732bc3c13be9c196f7de50defe195"
server_key_version: "encryptionkey@0"
I20260812 14:18:18.419011 17602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "aa0b5916c8ad44a191e11342b26c076d"
format_stamp: "Formatted at 2026-08-12 14:18:18 on dist-test-slave-5k54"
server_key: "feb2f83786aff83d7d0d6d240eb2f18a"
server_key_iv: "085732bc3c13be9c196f7de50defe195"
server_key_version: "encryptionkey@0"
I20260812 14:18:18.420341 17602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:18:18.421033 17621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:18.421224 17602 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:18.421309 17602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "aa0b5916c8ad44a191e11342b26c076d"
format_stamp: "Formatted at 2026-08-12 14:18:18 on dist-test-slave-5k54"
server_key: "feb2f83786aff83d7d0d6d240eb2f18a"
server_key_iv: "085732bc3c13be9c196f7de50defe195"
server_key_version: "encryptionkey@0"
I20260812 14:18:18.421363 17602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:18.455963 17602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:18.458287 17602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:18.458412 17602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:18.458876 17602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:18.458916 17602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:18.458940 17602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:18.458952 17602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:18.466405 17602 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:37335
I20260812 14:18:18.466482 17758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:37335 every 8 connection(s)
Aug 12 14:18:18 dist-test-slave-5k54 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544298, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
I20260812 14:18:18.466801 17602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:18:18.467446 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 17602
I20260812 14:18:18.467563 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:18:18.467787 8532 external_mini_cluster.cc:1428] Setting key d498d21dac85d2175727470e2498dba0
I20260812 14:18:18.470356 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.459554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:34595 (local address 127.8.85.62:33823)
0812 14:18:18.459633 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.459636 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.459916 (+ 280us) server_negotiation.cc:407] Connection header received
0812 14:18:18.460779 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.460780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.460799 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.460898 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:18.461877 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.462703 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.464432 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.464629 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.464688 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.467440 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:18.467442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:18.467442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:18.467450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:18.468955 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.469366 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.469368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.469368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.469438 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.469697 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.469698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.469698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.469814 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:18.469890 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.470179 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.470266 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 14:18:18.470573 17633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.459536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33823 (local address 127.8.85.2:34595)
0812 14:18:18.459745 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:18.459800 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:18.460584 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:18.461012 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:18.461018 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:18.461435 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:18.461749 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.461756 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.462810 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.462811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.464344 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.464347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.464717 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.464718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.464908 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.465538 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:18.465550 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:18.467296 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:18.469043 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:18.469050 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:18.469053 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:18.469285 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:18.469508 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:18.469509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:18.469509 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:18.469623 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:18.469881 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:18.469887 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:18.470044 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 14:18:18.470156 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 14:18:18.470317 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":105,"threads_started":1}
I20260812 14:18:18.471060 17760 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33823
I20260812 14:18:18.471153 17760 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:18.471349 17760 heartbeater.cc:511] Master 127.8.85.62:33823 requested a full tablet report, sending...
I20260812 14:18:18.471763 17349 ts_manager.cc:194] Registered new tserver with Master: aa0b5916c8ad44a191e11342b26c076d (127.8.85.2:37335)
I20260812 14:18:18.472114 17349 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:34595
I20260812 14:18:18.478368 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.472296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:35919 (local address 127.8.85.62:33823)
0812 14:18:18.472332 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.472334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.472510 (+ 176us) server_negotiation.cc:407] Connection header received
0812 14:18:18.472556 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.472557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.472568 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.472627 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:18.473023 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.473464 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.474568 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.474741 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.474795 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.475852 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:18.475854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:18.475854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:18.475860 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:18.477196 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.477599 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.477600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.477600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.477649 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.477853 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.477853 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.477853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.478004 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:18.478059 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.478247 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.478307 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:18:18.478480 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.472152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33823 (local address 127.8.85.2:35919)
0812 14:18:18.472429 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:18.472440 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:18.472503 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:18.472630 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:18.472631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:18.472864 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:18.472930 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.472931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.473575 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.473576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.474489 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.474492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.474817 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.474818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.474898 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.475294 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:18.475300 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:18.475769 (+ 469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:18.477282 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:18.477284 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:18.477284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:18.477515 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:18.477714 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:18.477715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:18.477715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:18.477789 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:18.478077 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:18.478078 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:18.478132 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 14:18:18.478221 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 14:18:18.478323 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":63,"threads_started":1}
WARNING: no policy specified for oryx/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.3.keytab.
Entry for principal oryx/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.3.keytab.
I20260812 14:18:18.497910 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33823
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.3.keytab
--principal=oryx/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:18.581959 17769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:18.582188 17769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:18.582230 17769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:18.583580 17769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:18.583640 17769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:18.583675 17769 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:18:18.583711 17769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:18:18.585222 17769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.3.keytab
--principal=oryx/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.85.62:33823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:18.585497 17769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:18.585729 17769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:18.590759 17775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:18.590803 17776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:18.590770 17778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:18.591100 17769 server_base.cc:1034] running on GCE node
I20260812 14:18:18.591287 17769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:18.591563 17769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:18.592705 17769 hybrid_clock.cc:648] HybridClock initialized: now 1786544298592695 us; error 25 us; skew 500 ppm
Aug 12 14:18:18 dist-test-slave-5k54 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544298, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:18.595188 17769 init.cc:377] Logged in from keytab as oryx/127.8.85.3@KRBTEST.COM (short username oryx)
I20260812 14:18:18.595664 17769 webserver.cc:460] Webserver started at http://127.8.85.3:36259/ using document root <none> and password file <none>
I20260812 14:18:18.595862 17769 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:18.595911 17769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:18.596007 17769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:18.597002 17769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "679da97817a24619b9fea715485a2211"
format_stamp: "Formatted at 2026-08-12 14:18:18 on dist-test-slave-5k54"
server_key: "433112935e5fd409a4d7cf13c11c6f63"
server_key_iv: "363f289c1148e900b15fef8aea629f40"
server_key_version: "encryptionkey@0"
I20260812 14:18:18.597304 17769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "679da97817a24619b9fea715485a2211"
format_stamp: "Formatted at 2026-08-12 14:18:18 on dist-test-slave-5k54"
server_key: "433112935e5fd409a4d7cf13c11c6f63"
server_key_iv: "363f289c1148e900b15fef8aea629f40"
server_key_version: "encryptionkey@0"
I20260812 14:18:18.598599 17769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:18:18.599295 17785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:18.599506 17769 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:18.599593 17769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "679da97817a24619b9fea715485a2211"
format_stamp: "Formatted at 2026-08-12 14:18:18 on dist-test-slave-5k54"
server_key: "433112935e5fd409a4d7cf13c11c6f63"
server_key_iv: "363f289c1148e900b15fef8aea629f40"
server_key_version: "encryptionkey@0"
I20260812 14:18:18.599648 17769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:18.614074 17769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:18.615900 17769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:18.616011 17769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:18.616457 17769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:18.616499 17769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:18.616523 17769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:18.616560 17769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:18.623775 17769 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:41515
I20260812 14:18:18.623850 17924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:41515 every 8 connection(s)
Aug 12 14:18:18 dist-test-slave-5k54 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544298, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
I20260812 14:18:18.624235 17769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:18:18.627290 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.616957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:42039 (local address 127.8.85.62:33823)
0812 14:18:18.617031 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.617034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.617219 (+ 185us) server_negotiation.cc:407] Connection header received
0812 14:18:18.617776 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.617777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.617790 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.617865 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:18.618827 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.619653 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.621325 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.621657 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.621728 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.624686 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:18.624688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:18.624689 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:18.624698 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:18.625988 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.626371 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.626372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.626373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.626437 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.626661 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.626662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.626662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.626777 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:18.626835 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.627098 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.627194 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 14:18:18.627480 17795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.616933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33823 (local address 127.8.85.3:42039)
0812 14:18:18.617109 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:18.617148 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:18.617666 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:18.617994 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:18.617999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:18.618407 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:18.618713 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.618721 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.619757 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.619758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.621221 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.621224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.621751 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.621752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.621930 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.622537 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:18.622548 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:18.624570 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:18.626062 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:18.626068 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:18.626070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:18.626282 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:18.626508 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:18.626509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:18.626509 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:18.626605 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:18.626828 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:18.626830 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:18.626931 (+ 101us) client_negotiation.cc:769] Sending connection context
0812 14:18:18.627071 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 14:18:18.627229 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260812 14:18:18.627976 17925 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33823
I20260812 14:18:18.628088 17925 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:18.628270 17925 heartbeater.cc:511] Master 127.8.85.62:33823 requested a full tablet report, sending...
I20260812 14:18:18.628729 17349 ts_manager.cc:194] Registered new tserver with Master: 679da97817a24619b9fea715485a2211 (127.8.85.3:41515)
I20260812 14:18:18.629243 17349 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:42039
I20260812 14:18:18.633387 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 17769
I20260812 14:18:18.633481 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:18:18.633705 8532 external_mini_cluster.cc:1428] Setting key 691b38b97475fe238efde539eb364549
I20260812 14:18:18.634613 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:18:18.635008 17931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.629014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33823 (local address 127.8.85.3:46125)
0812 14:18:18.629142 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:18.629154 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:18.629213 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:18.629338 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:18.629339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:18.629538 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:18.629603 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.629605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.630232 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.630233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.630755 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.630756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.631124 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.631125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.631193 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.631709 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:18.631714 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:18.632000 (+ 286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:18.633850 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:18.633851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:18.633852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:18.634107 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:18.634313 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:18.634314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:18.634314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:18.634398 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:18.634749 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:18.634750 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:18.634796 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 14:18:18.634859 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 14:18:18.634937 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
I20260812 14:18:18.635062 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.629089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:46125 (local address 127.8.85.62:33823)
0812 14:18:18.629133 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.629135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.629197 (+ 62us) server_negotiation.cc:407] Connection header received
0812 14:18:18.629276 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.629277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.629286 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.629336 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:18.629666 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.630135 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.630849 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.631051 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.631123 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.632108 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:18.632109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:18.632109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:18.632116 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:18.633749 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.634181 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.634182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.634182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.634244 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.634488 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.634489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.634489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.634628 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:18.634697 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.634908 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.634982 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Aug 12 14:18:18 dist-test-slave-5k54 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544298, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 14:18:18 dist-test-slave-5k54 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:18 dist-test-slave-5k54 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 14:18:18.656898 17412 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:18.651486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52242 (local address 127.8.85.62:33823)
0812 14:18:18.651545 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.651548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.651558 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:18:18.651607 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.651607 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.651625 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.651687 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:18.652264 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.652857 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.653648 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.653840 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.653931 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.656807 (+ 2876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52242: BlockingRecv error: recv got EOF from 127.0.0.1:52242 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 12 14:18:18 dist-test-slave-5k54 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544298, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
I20260812 14:18:18.664018 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.657642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52248 (local address 127.8.85.62:33823)
0812 14:18:18.657701 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.657703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.657752 (+ 49us) server_negotiation.cc:407] Connection header received
0812 14:18:18.657838 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.657839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.657849 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.657930 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:18.658385 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.658822 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.659551 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.659735 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.659808 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.661689 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:18.661691 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:18.661691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:18.661698 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:18.662804 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.663204 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.663205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.663205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.663265 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.663515 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.663516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.663516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.663626 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:18.663725 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.663844 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.663924 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:18:18.665709 17349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:18:18.667743 17349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:18:18.674259 17950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.670007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:37335 (local address 127.0.0.1:39426)
0812 14:18:18.670261 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:18.670271 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:18.670311 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:18.670992 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:18.670992 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:18.670995 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:18.671059 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.671060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.672111 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.672112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.673180 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.673183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.673938 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.673939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.674026 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.674041 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 14:18:18.674119 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 14:18:18.674170 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":1,"spinlock_wait_cycles":9600,"thread_start_us":110,"threads_started":1}
I20260812 14:18:18.674290 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.670266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39426 (local address 127.8.85.2:37335)
0812 14:18:18.670768 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.670773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.670783 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:18:18.670820 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.670822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.670837 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.670912 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:18.671160 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.671996 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.673266 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.673847 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.673921 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.673993 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.674155 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.674189 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":75,"threads_started":1}
I20260812 14:18:18.674654 17949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.670007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:41515 (local address 127.0.0.1:50438)
0812 14:18:18.670403 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:18.670412 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:18.670458 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:18.670715 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:18.670716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:18.670718 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:18.670775 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.670777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.672150 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.672151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.673524 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.673526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.674448 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.674449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.674501 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.674505 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:18.674538 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 14:18:18.674589 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":142,"thread_start_us":79,"threads_started":1}
I20260812 14:18:18.675056 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.670236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50438 (local address 127.8.85.3:41515)
0812 14:18:18.670441 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.670445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.670455 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:18:18.670530 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.670532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.670544 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.670647 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:18.670857 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.672016 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.673626 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.674238 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.674303 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.674900 (+ 597us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.674928 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.674966 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":97,"threads_started":1}
I20260812 14:18:18.675154 17951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.669974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:42667 (local address 127.0.0.1:47784)
0812 14:18:18.670173 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:18.670214 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:18.670286 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:18.670616 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:18.670620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:18.670623 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:18.670708 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.670710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.672157 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.672158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.673522 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.673526 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.674725 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.674726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.674964 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.674971 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 14:18:18.675020 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:18:18.675064 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":95,"thread_start_us":66,"threads_started":1}
I20260812 14:18:18.675071 17681 tablet_service.cc:1467] Processing CreateTablet for tablet 0dbbf4706ce6464a8b6c9b5115bbab9d (DEFAULT_TABLE table=test-table [id=5e30526afbd94f8c9e93978edb5a1ae1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:18.675518 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.670084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47784 (local address 127.8.85.1:42667)
0812 14:18:18.670440 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.670445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.670456 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:18:18.670496 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.670498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.670515 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.670628 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:18.670800 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.672055 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.673612 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.674620 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.674791 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.674882 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.675323 (+ 441us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.675388 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":83,"threads_started":1}
I20260812 14:18:18.675520 17681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dbbf4706ce6464a8b6c9b5115bbab9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:18.675829 17842 tablet_service.cc:1467] Processing CreateTablet for tablet 0dbbf4706ce6464a8b6c9b5115bbab9d (DEFAULT_TABLE table=test-table [id=5e30526afbd94f8c9e93978edb5a1ae1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:18.676190 17842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dbbf4706ce6464a8b6c9b5115bbab9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:18.676182 17516 tablet_service.cc:1467] Processing CreateTablet for tablet 0dbbf4706ce6464a8b6c9b5115bbab9d (DEFAULT_TABLE table=test-table [id=5e30526afbd94f8c9e93978edb5a1ae1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:18.676529 17516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dbbf4706ce6464a8b6c9b5115bbab9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:18.679104 17955 tablet_bootstrap.cc:492] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211: Bootstrap starting.
I20260812 14:18:18.679867 17956 tablet_bootstrap.cc:492] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d: Bootstrap starting.
I20260812 14:18:18.680001 17955 tablet_bootstrap.cc:654] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:18.680359 17955 log.cc:824] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:18.680495 17958 tablet_bootstrap.cc:492] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb: Bootstrap starting.
I20260812 14:18:18.680536 17956 tablet_bootstrap.cc:654] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:18.680864 17956 log.cc:824] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:18.681149 17958 tablet_bootstrap.cc:654] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:18.681372 17955 tablet_bootstrap.cc:492] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211: No bootstrap required, opened a new log
I20260812 14:18:18.681460 17955 ts_tablet_manager.cc:1397] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 14:18:18.681521 17958 log.cc:824] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:18.681661 17956 tablet_bootstrap.cc:492] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d: No bootstrap required, opened a new log
I20260812 14:18:18.681731 17956 ts_tablet_manager.cc:1397] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:18:18.683532 17955 raft_consensus.cc:348] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.683709 17955 raft_consensus.cc:374] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:18.683746 17955 raft_consensus.cc:729] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 679da97817a24619b9fea715485a2211, State: Initialized, Role: FOLLOWER
I20260812 14:18:18.683867 17955 consensus_queue.cc:260] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.684016 17958 tablet_bootstrap.cc:492] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb: No bootstrap required, opened a new log
I20260812 14:18:18.683919 17956 raft_consensus.cc:348] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.684085 17958 ts_tablet_manager.cc:1397] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 14:18:18.684082 17956 raft_consensus.cc:374] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:18.684118 17956 raft_consensus.cc:729] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0b5916c8ad44a191e11342b26c076d, State: Initialized, Role: FOLLOWER
I20260812 14:18:18.684152 17955 ts_tablet_manager.cc:1428] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 14:18:18.684202 17925 heartbeater.cc:503] Master 127.8.85.62:33823 was elected leader, sending a full tablet report...
I20260812 14:18:18.684247 17956 consensus_queue.cc:260] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.684520 17956 ts_tablet_manager.cc:1428] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:18:18.684947 17760 heartbeater.cc:503] Master 127.8.85.62:33823 was elected leader, sending a full tablet report...
I20260812 14:18:18.686029 17958 raft_consensus.cc:348] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.686142 17958 raft_consensus.cc:374] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:18.686162 17958 raft_consensus.cc:729] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32284027752745229022fbd48077dcbb, State: Initialized, Role: FOLLOWER
I20260812 14:18:18.686241 17958 consensus_queue.cc:260] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.686431 17958 ts_tablet_manager.cc:1428] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:18:18.686527 17593 heartbeater.cc:503] Master 127.8.85.62:33823 was elected leader, sending a full tablet report...
W20260812 14:18:18.717571 17761 tablet.cc:2369] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:18:18.785467 17594 tablet.cc:2369] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:18:18.806720 17966 raft_consensus.cc:482] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:18:18.806850 17966 raft_consensus.cc:504] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.807211 17966 leader_election.cc:288] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa0b5916c8ad44a191e11342b26c076d (127.8.85.2:37335), 679da97817a24619b9fea715485a2211 (127.8.85.3:41515)
I20260812 14:18:18.810278 17968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.807481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:37335 (local address 127.8.85.1:50879)
0812 14:18:18.807632 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:18.807644 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:18.807722 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:18.807892 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:18.807893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:18.807895 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:18.807978 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.807980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.808646 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.808647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.809456 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.809458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.810008 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.810009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.810082 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.810087 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:18:18.810138 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:18:18.810179 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":116,"threads_started":1}
I20260812 14:18:18.810298 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.807581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:50879 (local address 127.8.85.2:37335)
0812 14:18:18.807635 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.807637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.807725 (+ 88us) server_negotiation.cc:407] Connection header received
0812 14:18:18.807802 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.807804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.807817 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.807878 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:18.808057 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.808564 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.809541 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.809927 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.809982 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.809990 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.810161 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.810207 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:18:18.810359 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.807600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:44581 (local address 127.8.85.3:41515)
0812 14:18:18.807646 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.807648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.807764 (+ 116us) server_negotiation.cc:407] Connection header received
0812 14:18:18.807889 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.807890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.807902 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.807968 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:18.808146 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.808587 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.809620 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.809996 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.810067 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.810078 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.810247 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.810285 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:18:18.810428 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.807493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:41515 (local address 127.8.85.1:44581)
0812 14:18:18.807675 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:18.807686 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:18.807760 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:18.807989 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:18.807990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:18.807992 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:18.808075 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.808077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.808700 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.808701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.809533 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.809536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.810082 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.810083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.810167 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.810171 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:18.810228 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 14:18:18.810269 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":68,"threads_started":1}
I20260812 14:18:18.810678 17704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dbbf4706ce6464a8b6c9b5115bbab9d" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0b5916c8ad44a191e11342b26c076d" is_pre_election: true
I20260812 14:18:18.810837 17704 raft_consensus.cc:2454] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 0.
I20260812 14:18:18.810807 17864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dbbf4706ce6464a8b6c9b5115bbab9d" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679da97817a24619b9fea715485a2211" is_pre_election: true
I20260812 14:18:18.810961 17864 raft_consensus.cc:2454] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 0.
I20260812 14:18:18.811111 17456 leader_election.cc:302] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32284027752745229022fbd48077dcbb, aa0b5916c8ad44a191e11342b26c076d; no voters:
I20260812 14:18:18.811262 17966 raft_consensus.cc:2781] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:18:18.811364 17966 raft_consensus.cc:482] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:18:18.811401 17966 raft_consensus.cc:3037] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:18.812219 17966 raft_consensus.cc:504] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.812376 17966 leader_election.cc:288] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 election: Requested vote from peers aa0b5916c8ad44a191e11342b26c076d (127.8.85.2:37335), 679da97817a24619b9fea715485a2211 (127.8.85.3:41515)
I20260812 14:18:18.812537 17864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dbbf4706ce6464a8b6c9b5115bbab9d" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679da97817a24619b9fea715485a2211"
I20260812 14:18:18.812561 17704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dbbf4706ce6464a8b6c9b5115bbab9d" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0b5916c8ad44a191e11342b26c076d"
I20260812 14:18:18.812635 17864 raft_consensus.cc:3037] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:18.812633 17704 raft_consensus.cc:3037] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:18.813333 17704 raft_consensus.cc:2454] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 1.
I20260812 14:18:18.813413 17864 raft_consensus.cc:2454] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 1.
I20260812 14:18:18.813493 17456 leader_election.cc:302] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32284027752745229022fbd48077dcbb, aa0b5916c8ad44a191e11342b26c076d; no voters:
I20260812 14:18:18.813580 17966 raft_consensus.cc:2781] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:18.813688 17966 raft_consensus.cc:686] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [term 1 LEADER]: Becoming Leader. State: Replica: 32284027752745229022fbd48077dcbb, State: Running, Role: LEADER
I20260812 14:18:18.813800 17966 consensus_queue.cc:237] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.814631 17348 catalog_manager.cc:5495] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb reported cstate change: term changed from 0 to 1, leader changed from <none> to 32284027752745229022fbd48077dcbb (127.8.85.1). New cstate: current_term: 1 leader_uuid: "32284027752745229022fbd48077dcbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } health_report { overall_health: UNKNOWN } } }
I20260812 14:18:18.855427 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.853241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47796 (local address 127.8.85.1:42667)
0812 14:18:18.853287 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.853289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.853297 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:18:18.853378 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.853379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.853400 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.853459 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:18:18.853572 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.854031 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.854741 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.854916 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.854976 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.855104 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:18:18.855199 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:18:18.855251 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.855297 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.855313 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:18:18.857442 17704 raft_consensus.cc:1261] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Refusing update from remote peer 32284027752745229022fbd48077dcbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:18.857448 17864 raft_consensus.cc:1261] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Refusing update from remote peer 32284027752745229022fbd48077dcbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:18.857834 17970 consensus_queue.cc:1035] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:18.858000 17966 consensus_queue.cc:1035] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:18.860179 17974 mvcc.cc:205] Tried to move back new op lower bound from 7317685448116830208 to 7317685447941980160. Current Snapshot: MvccSnapshot[applied={T|T < 7317685448116830208}]
I20260812 14:18:18.861202 17975 mvcc.cc:205] Tried to move back new op lower bound from 7317685448116830208 to 7317685447941980160. Current Snapshot: MvccSnapshot[applied={T|T < 7317685448116830208}]
I20260812 14:18:18.862890 17349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:18.862970 17349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:18.863767 17349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215: Sending DeleteTablet for 3 replicas of tablet 0dbbf4706ce6464a8b6c9b5115bbab9d
I20260812 14:18:18.864104 17516 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dbbf4706ce6464a8b6c9b5115bbab9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:18 UTC) from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:47784
I20260812 14:18:18.864153 17842 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dbbf4706ce6464a8b6c9b5115bbab9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:18 UTC) from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:50438
I20260812 14:18:18.864259 17681 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dbbf4706ce6464a8b6c9b5115bbab9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:18 UTC) from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:39426
I20260812 14:18:18.864452 17986 tablet_replica.cc:330] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb: stopping tablet replica
I20260812 14:18:18.864562 17986 raft_consensus.cc:2229] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:18:18.864519 17349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:18:18.864660 17349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:18:18.864684 17986 raft_consensus.cc:2258] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:18.865105 17987 tablet_replica.cc:330] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211: stopping tablet replica
I20260812 14:18:18.865262 17987 raft_consensus.cc:2229] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:18.865446 17987 raft_consensus.cc:2258] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:18.865975 17681 tablet_service.cc:1467] Processing CreateTablet for tablet 8f23ab28123b49ccb83cf8afa02718d0 (DEFAULT_TABLE table=test-table [id=fbe25ea18639475a8a1bb44672d21bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:18.865978 17516 tablet_service.cc:1467] Processing CreateTablet for tablet 8f23ab28123b49ccb83cf8afa02718d0 (DEFAULT_TABLE table=test-table [id=fbe25ea18639475a8a1bb44672d21bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:18.866106 17681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f23ab28123b49ccb83cf8afa02718d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:18.866106 17516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f23ab28123b49ccb83cf8afa02718d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:18.866010 17987 ts_tablet_manager.cc:1905] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:18.866096 17842 tablet_service.cc:1467] Processing CreateTablet for tablet 8f23ab28123b49ccb83cf8afa02718d0 (DEFAULT_TABLE table=test-table [id=fbe25ea18639475a8a1bb44672d21bb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:18.866325 17842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f23ab28123b49ccb83cf8afa02718d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:18.867658 17956 tablet_bootstrap.cc:492] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d: Bootstrap starting.
I20260812 14:18:18.867678 17958 tablet_bootstrap.cc:492] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb: Bootstrap starting.
I20260812 14:18:18.868091 17956 tablet_bootstrap.cc:654] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:18.868161 17958 tablet_bootstrap.cc:654] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:18.868371 17987 ts_tablet_manager.cc:1918] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:18.868441 17987 log.cc:1196] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/0dbbf4706ce6464a8b6c9b5115bbab9d
I20260812 14:18:18.868723 17987 ts_tablet_manager.cc:1939] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 679da97817a24619b9fea715485a2211: Deleting consensus metadata
I20260812 14:18:18.869216 17986 ts_tablet_manager.cc:1905] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:18.871171 17986 ts_tablet_manager.cc:1918] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:18.871294 17986 log.cc:1196] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/wal/wals/0dbbf4706ce6464a8b6c9b5115bbab9d
I20260812 14:18:18.871630 17986 ts_tablet_manager.cc:1939] T 0dbbf4706ce6464a8b6c9b5115bbab9d P 32284027752745229022fbd48077dcbb: Deleting consensus metadata
I20260812 14:18:18.872144 17333 catalog_manager.cc:4841] TS 32284027752745229022fbd48077dcbb (127.8.85.1:42667): tablet 0dbbf4706ce6464a8b6c9b5115bbab9d (table test-table [id=5e30526afbd94f8c9e93978edb5a1ae1]) successfully deleted
I20260812 14:18:18.872371 17956 tablet_bootstrap.cc:492] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d: No bootstrap required, opened a new log
I20260812 14:18:18.872447 17956 ts_tablet_manager.cc:1397] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 14:18:18.872632 17956 raft_consensus.cc:348] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.872733 17956 raft_consensus.cc:374] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:18.872759 17956 raft_consensus.cc:729] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0b5916c8ad44a191e11342b26c076d, State: Initialized, Role: FOLLOWER
I20260812 14:18:18.872814 17956 consensus_queue.cc:260] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.872924 17956 ts_tablet_manager.cc:1428] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:18.874681 17955 tablet_bootstrap.cc:492] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211: Bootstrap starting.
I20260812 14:18:18.875298 17955 tablet_bootstrap.cc:654] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:18.876834 17334 catalog_manager.cc:4841] TS 679da97817a24619b9fea715485a2211 (127.8.85.3:41515): tablet 0dbbf4706ce6464a8b6c9b5115bbab9d (table test-table [id=5e30526afbd94f8c9e93978edb5a1ae1]) successfully deleted
I20260812 14:18:18.876988 17988 tablet_replica.cc:330] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d: stopping tablet replica
I20260812 14:18:18.877105 17988 raft_consensus.cc:2229] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:18.877228 17988 raft_consensus.cc:2258] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:18.877704 17988 ts_tablet_manager.cc:1905] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:18.879509 17988 ts_tablet_manager.cc:1918] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:18.879588 17988 log.cc:1196] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/0dbbf4706ce6464a8b6c9b5115bbab9d
I20260812 14:18:18.879853 17988 ts_tablet_manager.cc:1939] T 0dbbf4706ce6464a8b6c9b5115bbab9d P aa0b5916c8ad44a191e11342b26c076d: Deleting consensus metadata
I20260812 14:18:18.880295 17335 catalog_manager.cc:4841] TS aa0b5916c8ad44a191e11342b26c076d (127.8.85.2:37335): tablet 0dbbf4706ce6464a8b6c9b5115bbab9d (table test-table [id=5e30526afbd94f8c9e93978edb5a1ae1]) successfully deleted
I20260812 14:18:18.880357 17955 tablet_bootstrap.cc:492] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211: No bootstrap required, opened a new log
I20260812 14:18:18.880401 17955 ts_tablet_manager.cc:1397] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.000s
I20260812 14:18:18.880548 17955 raft_consensus.cc:348] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.880621 17955 raft_consensus.cc:374] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:18.880640 17955 raft_consensus.cc:729] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 679da97817a24619b9fea715485a2211, State: Initialized, Role: FOLLOWER
I20260812 14:18:18.880689 17955 consensus_queue.cc:260] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.880784 17955 ts_tablet_manager.cc:1428] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:18.881223 17958 tablet_bootstrap.cc:492] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb: No bootstrap required, opened a new log
I20260812 14:18:18.881274 17958 ts_tablet_manager.cc:1397] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20260812 14:18:18.881410 17958 raft_consensus.cc:348] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.881481 17958 raft_consensus.cc:374] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:18.881500 17958 raft_consensus.cc:729] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32284027752745229022fbd48077dcbb, State: Initialized, Role: FOLLOWER
I20260812 14:18:18.881546 17958 consensus_queue.cc:260] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.881639 17958 ts_tablet_manager.cc:1428] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:18.946727 17965 raft_consensus.cc:482] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:18:18.946825 17965 raft_consensus.cc:504] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.947153 17965 leader_election.cc:288] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32284027752745229022fbd48077dcbb (127.8.85.1:42667), 679da97817a24619b9fea715485a2211 (127.8.85.3:41515)
I20260812 14:18:18.950675 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.947404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:59833 (local address 127.8.85.3:41515)
0812 14:18:18.947474 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.947476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.947584 (+ 108us) server_negotiation.cc:407] Connection header received
0812 14:18:18.947709 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.947710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.947720 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.947765 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:18.947972 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.948675 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.949945 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.950311 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.950397 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.950408 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.950564 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.950603 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 14:18:18.950703 17994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.947339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:41515 (local address 127.8.85.2:59833)
0812 14:18:18.947524 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:18.947536 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:18.947616 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:18.947777 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:18.947778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:18.947779 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:18.947878 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.947880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.948772 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.948773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.949860 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.949861 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.950399 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.950400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.950487 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.950491 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:18.950558 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 14:18:18.950600 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":61,"thread_start_us":82,"threads_started":1}
I20260812 14:18:18.950865 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.947620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:35899 (local address 127.8.85.1:42667)
0812 14:18:18.947703 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.947705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.947717 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:18:18.947754 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.947755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.947765 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.947838 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:18.947977 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.948445 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.949826 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.950375 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.950451 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.950463 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.950744 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.950800 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 14:18:18.950968 17864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f23ab28123b49ccb83cf8afa02718d0" candidate_uuid: "aa0b5916c8ad44a191e11342b26c076d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679da97817a24619b9fea715485a2211" is_pre_election: true
I20260812 14:18:18.951061 17633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.947335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:42667 (local address 127.8.85.2:35899)
0812 14:18:18.947390 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:18.947403 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:18.947469 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:18.947840 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:18.947841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:18.947843 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:18.947885 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.947886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.948533 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.948534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.949710 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.949713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.950501 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.950502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.950588 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.950592 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:18.950643 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:18:18.950691 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 14:18:18.951068 17864 raft_consensus.cc:2454] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa0b5916c8ad44a191e11342b26c076d in term 0.
I20260812 14:18:18.951231 17624 leader_election.cc:302] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 679da97817a24619b9fea715485a2211, aa0b5916c8ad44a191e11342b26c076d; no voters:
I20260812 14:18:18.951339 17965 raft_consensus.cc:2781] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:18:18.951370 17965 raft_consensus.cc:482] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:18:18.951395 17965 raft_consensus.cc:3037] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:18.951330 17539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f23ab28123b49ccb83cf8afa02718d0" candidate_uuid: "aa0b5916c8ad44a191e11342b26c076d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32284027752745229022fbd48077dcbb" is_pre_election: true
I20260812 14:18:18.951483 17539 raft_consensus.cc:2454] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa0b5916c8ad44a191e11342b26c076d in term 0.
I20260812 14:18:18.952215 17965 raft_consensus.cc:504] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.952363 17965 leader_election.cc:288] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [CANDIDATE]: Term 1 election: Requested vote from peers 32284027752745229022fbd48077dcbb (127.8.85.1:42667), 679da97817a24619b9fea715485a2211 (127.8.85.3:41515)
I20260812 14:18:18.952582 17864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f23ab28123b49ccb83cf8afa02718d0" candidate_uuid: "aa0b5916c8ad44a191e11342b26c076d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679da97817a24619b9fea715485a2211"
I20260812 14:18:18.952584 17539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f23ab28123b49ccb83cf8afa02718d0" candidate_uuid: "aa0b5916c8ad44a191e11342b26c076d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32284027752745229022fbd48077dcbb"
I20260812 14:18:18.952661 17539 raft_consensus.cc:3037] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:18.952682 17864 raft_consensus.cc:3037] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:18.953538 17864 raft_consensus.cc:2454] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa0b5916c8ad44a191e11342b26c076d in term 1.
I20260812 14:18:18.953565 17539 raft_consensus.cc:2454] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa0b5916c8ad44a191e11342b26c076d in term 1.
I20260812 14:18:18.953738 17624 leader_election.cc:302] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 679da97817a24619b9fea715485a2211, aa0b5916c8ad44a191e11342b26c076d; no voters:
I20260812 14:18:18.953871 17965 raft_consensus.cc:2781] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:18.954021 17965 raft_consensus.cc:686] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [term 1 LEADER]: Becoming Leader. State: Replica: aa0b5916c8ad44a191e11342b26c076d, State: Running, Role: LEADER
I20260812 14:18:18.954116 17965 consensus_queue.cc:237] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.954866 17349 catalog_manager.cc:5495] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d reported cstate change: term changed from 0 to 1, leader changed from <none> to aa0b5916c8ad44a191e11342b26c076d (127.8.85.2). New cstate: current_term: 1 leader_uuid: "aa0b5916c8ad44a191e11342b26c076d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } health_report { overall_health: UNKNOWN } } }
Aug 12 14:18:18 dist-test-slave-5k54 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544298, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.62@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
I20260812 14:18:18.992149 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.984220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52264 (local address 127.8.85.62:33823)
0812 14:18:18.984272 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:18.984275 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:18.984392 (+ 117us) server_negotiation.cc:407] Connection header received
0812 14:18:18.984464 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:18.984464 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:18.984474 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:18.984526 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:18.985129 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.985611 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.986782 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.986966 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.987014 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.989464 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:18.989466 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:18.989466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:18.989472 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:18.990878 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.991283 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.991283 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.991284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.991336 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:18.991549 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:18.991549 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:18.991549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:18.991650 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:18.991703 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:18.991921 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 14:18:18.992060 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":28}
I20260812 14:18:18.992156 18003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:18.984121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33823 (local address 127.0.0.1:52264)
0812 14:18:18.984305 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:18.984318 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:18.984400 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:18.984558 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:18.984559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:18.984989 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:18.985051 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.985054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.985692 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.985693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.986682 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:18.986686 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:18.987046 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:18.987048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:18.987110 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:18.987767 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:18.987773 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:18.989382 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:18.990954 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:18.990961 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:18.990964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:18.991211 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:18.991398 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:18.991398 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:18.991399 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:18.991487 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:18.991711 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:18.991718 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:18.991768 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 14:18:18.991896 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 14:18:18.992015 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":91,"threads_started":1}
I20260812 14:18:18.994256 17349 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:52264:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 14:18:18.994477 17349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:18:18.996042 17516 tablet_service.cc:1467] Processing CreateTablet for tablet 60b9a0fecb9c4683a86f67484787bc91 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ac26d32fe8f0497ea560dd25721227ce]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:18:18.996088 17842 tablet_service.cc:1467] Processing CreateTablet for tablet 60b9a0fecb9c4683a86f67484787bc91 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ac26d32fe8f0497ea560dd25721227ce]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:18:18.996212 17516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60b9a0fecb9c4683a86f67484787bc91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:18.996270 17842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60b9a0fecb9c4683a86f67484787bc91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:18.997349 17958 tablet_bootstrap.cc:492] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb: Bootstrap starting.
I20260812 14:18:18.997429 17955 tablet_bootstrap.cc:492] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211: Bootstrap starting.
I20260812 14:18:18.997462 17681 tablet_service.cc:1467] Processing CreateTablet for tablet 60b9a0fecb9c4683a86f67484787bc91 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ac26d32fe8f0497ea560dd25721227ce]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:18:18.997598 17681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60b9a0fecb9c4683a86f67484787bc91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:18.997709 17958 tablet_bootstrap.cc:654] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:18.997855 17955 tablet_bootstrap.cc:654] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:18.998355 17958 tablet_bootstrap.cc:492] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb: No bootstrap required, opened a new log
I20260812 14:18:18.998411 17958 ts_tablet_manager.cc:1397] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:18.998533 17958 raft_consensus.cc:348] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.998625 17958 raft_consensus.cc:374] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:18.998653 17958 raft_consensus.cc:729] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32284027752745229022fbd48077dcbb, State: Initialized, Role: FOLLOWER
I20260812 14:18:18.998699 17958 consensus_queue.cc:260] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.998791 17958 ts_tablet_manager.cc:1428] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:18.998809 17966 tablet_replica.cc:439] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } } }
I20260812 14:18:18.998881 17966 tablet_replica.cc:442] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:18:18.999042 17956 tablet_bootstrap.cc:492] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d: Bootstrap starting.
I20260812 14:18:18.999302 17955 tablet_bootstrap.cc:492] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211: No bootstrap required, opened a new log
I20260812 14:18:18.999338 17955 ts_tablet_manager.cc:1397] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260812 14:18:18.999411 17956 tablet_bootstrap.cc:654] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:18.999541 17955 raft_consensus.cc:348] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.999593 17955 raft_consensus.cc:374] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:18.999606 17955 raft_consensus.cc:729] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 679da97817a24619b9fea715485a2211, State: Initialized, Role: FOLLOWER
I20260812 14:18:18.999678 17955 consensus_queue.cc:260] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:18.999787 17955 ts_tablet_manager.cc:1428] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:18.999934 17963 tablet_replica.cc:439] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } } }
I20260812 14:18:19.000037 17963 tablet_replica.cc:442] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:18:19.000216 17956 tablet_bootstrap.cc:492] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d: No bootstrap required, opened a new log
I20260812 14:18:19.000272 17956 ts_tablet_manager.cc:1397] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:18:19.000443 17956 raft_consensus.cc:348] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.000519 17956 raft_consensus.cc:374] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:19.000548 17956 raft_consensus.cc:729] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0b5916c8ad44a191e11342b26c076d, State: Initialized, Role: FOLLOWER
I20260812 14:18:19.000588 17956 consensus_queue.cc:260] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.000692 17956 ts_tablet_manager.cc:1428] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:19.000842 17965 tablet_replica.cc:439] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } } }
I20260812 14:18:19.000921 17965 tablet_replica.cc:442] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:18:19.015079 17966 raft_consensus.cc:482] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:18:19.015164 17966 raft_consensus.cc:504] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.015322 17966 leader_election.cc:288] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa0b5916c8ad44a191e11342b26c076d (127.8.85.2:37335), 679da97817a24619b9fea715485a2211 (127.8.85.3:41515)
I20260812 14:18:19.015565 17864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60b9a0fecb9c4683a86f67484787bc91" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679da97817a24619b9fea715485a2211" is_pre_election: true
I20260812 14:18:19.015604 17704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60b9a0fecb9c4683a86f67484787bc91" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0b5916c8ad44a191e11342b26c076d" is_pre_election: true
I20260812 14:18:19.015697 17704 raft_consensus.cc:2454] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 0.
I20260812 14:18:19.015700 17864 raft_consensus.cc:2454] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 0.
I20260812 14:18:19.015867 17455 leader_election.cc:302] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32284027752745229022fbd48077dcbb, 679da97817a24619b9fea715485a2211; no voters:
I20260812 14:18:19.015988 17966 raft_consensus.cc:2781] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:18:19.016037 17966 raft_consensus.cc:482] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:18:19.016065 17966 raft_consensus.cc:3037] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:19.016656 17966 raft_consensus.cc:504] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.016783 17966 leader_election.cc:288] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 election: Requested vote from peers aa0b5916c8ad44a191e11342b26c076d (127.8.85.2:37335), 679da97817a24619b9fea715485a2211 (127.8.85.3:41515)
I20260812 14:18:19.016901 17704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60b9a0fecb9c4683a86f67484787bc91" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0b5916c8ad44a191e11342b26c076d"
I20260812 14:18:19.016923 17864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60b9a0fecb9c4683a86f67484787bc91" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679da97817a24619b9fea715485a2211"
I20260812 14:18:19.016975 17704 raft_consensus.cc:3037] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:19.016994 17864 raft_consensus.cc:3037] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:19.017700 17864 raft_consensus.cc:2454] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 1.
I20260812 14:18:19.017724 17704 raft_consensus.cc:2454] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 1.
I20260812 14:18:19.017901 17455 leader_election.cc:302] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32284027752745229022fbd48077dcbb, 679da97817a24619b9fea715485a2211; no voters:
I20260812 14:18:19.018035 17966 raft_consensus.cc:2781] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:19.018093 17966 raft_consensus.cc:686] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [term 1 LEADER]: Becoming Leader. State: Replica: 32284027752745229022fbd48077dcbb, State: Running, Role: LEADER
I20260812 14:18:19.018149 17966 consensus_queue.cc:237] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.018471 17980 tablet_replica.cc:439] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb: TxnStatusTablet state changed. Reason: New leader 32284027752745229022fbd48077dcbb. Latest consensus state: current_term: 1 leader_uuid: "32284027752745229022fbd48077dcbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } } }
I20260812 14:18:19.018573 17980 tablet_replica.cc:442] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:18:19.018724 17349 catalog_manager.cc:5495] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb reported cstate change: term changed from 0 to 1, leader changed from <none> to 32284027752745229022fbd48077dcbb (127.8.85.1). New cstate: current_term: 1 leader_uuid: "32284027752745229022fbd48077dcbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } health_report { overall_health: UNKNOWN } } }
I20260812 14:18:19.019078 18008 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 14:18:19.019234 18008 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 14:18:19.029886 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:19.027490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47812 (local address 127.8.85.1:42667)
0812 14:18:19.027532 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:19.027534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:19.027591 (+ 57us) server_negotiation.cc:407] Connection header received
0812 14:18:19.027685 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:19.027686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:19.027695 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:19.027744 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:18:19.027884 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.028349 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.029157 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.029359 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.029437 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:19.029619 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:18:19.029661 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:18:19.029724 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:19.029796 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 14:18:19.029818 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:18:19.029934 18003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:19.027464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:42667 (local address 127.0.0.1:47812)
0812 14:18:19.027536 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:19.027549 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:19.027613 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:19.027767 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:19.027768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:19.027770 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:18:19.027824 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:19.027826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.028454 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.028455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:19.029063 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:19.029065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.029449 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.029450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:19.029521 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:19.029537 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:18:19.029741 (+ 204us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:18:19.029751 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 14:18:19.029793 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 14:18:19.029844 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 14:18:19.031006 17704 raft_consensus.cc:1261] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Refusing update from remote peer 32284027752745229022fbd48077dcbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:19.031081 17864 raft_consensus.cc:1261] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Refusing update from remote peer 32284027752745229022fbd48077dcbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:19.031255 17966 consensus_queue.cc:1035] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:19.031338 17980 consensus_queue.cc:1035] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:19.031934 17965 tablet_replica.cc:439] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d: TxnStatusTablet state changed. Reason: New leader 32284027752745229022fbd48077dcbb. Latest consensus state: current_term: 1 leader_uuid: "32284027752745229022fbd48077dcbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } } }
I20260812 14:18:19.031920 17963 tablet_replica.cc:439] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211: TxnStatusTablet state changed. Reason: New leader 32284027752745229022fbd48077dcbb. Latest consensus state: current_term: 1 leader_uuid: "32284027752745229022fbd48077dcbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } } }
I20260812 14:18:19.032032 17963 tablet_replica.cc:442] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:18:19.032029 17965 tablet_replica.cc:442] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:18:19.032649 17966 tablet_replica.cc:439] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32284027752745229022fbd48077dcbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } } }
I20260812 14:18:19.032723 17966 tablet_replica.cc:442] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:18:19.033057 17970 tablet_replica.cc:439] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32284027752745229022fbd48077dcbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } } }
I20260812 14:18:19.033094 17977 mvcc.cc:205] Tried to move back new op lower bound from 7317685448829333504 to 7317685448778678272. Current Snapshot: MvccSnapshot[applied={T|T < 7317685448829333504 or (T in {7317685448829333504})}]
I20260812 14:18:19.033150 17970 tablet_replica.cc:442] T 60b9a0fecb9c4683a86f67484787bc91 P 32284027752745229022fbd48077dcbb: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:18:19.033237 17963 tablet_replica.cc:439] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32284027752745229022fbd48077dcbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } } }
I20260812 14:18:19.033300 17963 tablet_replica.cc:442] T 60b9a0fecb9c4683a86f67484787bc91 P 679da97817a24619b9fea715485a2211: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:18:19.033541 17965 tablet_replica.cc:439] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32284027752745229022fbd48077dcbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } } }
I20260812 14:18:19.033627 17965 tablet_replica.cc:442] T 60b9a0fecb9c4683a86f67484787bc91 P aa0b5916c8ad44a191e11342b26c076d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:18:19.037313 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:19.034902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39432 (local address 127.8.85.2:37335)
0812 14:18:19.034939 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:19.034941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:19.034949 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:18:19.035013 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:19.035014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:19.035033 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:19.035116 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:18:19.035228 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.035758 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.036558 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.036777 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.036854 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:19.036987 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:18:19.037094 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:18:19.037172 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:19.037220 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 14:18:19.037240 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:18:19.042341 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:19.039809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:39691 (local address 127.8.85.1:42667)
0812 14:18:19.039856 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:19.039857 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:19.039997 (+ 140us) server_negotiation.cc:407] Connection header received
0812 14:18:19.040024 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:19.040025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:19.040038 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:19.040102 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:18:19.040203 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.040898 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.041612 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.041899 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.041977 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:19.042128 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:18:19.042159 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:18:19.042210 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:19.042255 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 14:18:19.042275 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:18:19.042539 18015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:19.039746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:42667 (local address 127.8.85.2:39691)
0812 14:18:19.039919 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:19.039930 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:19.039985 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:19.040103 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:19.040103 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:19.040104 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:18:19.040147 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:19.040148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.040975 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.040976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:19.041537 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:19.041538 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.041994 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.041995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:19.042038 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:19.042050 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:18:19.042217 (+ 167us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:18:19.042218 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 14:18:19.042252 (+ 34us) client_negotiation.cc:240] Negotiation successful
0812 14:18:19.042286 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
I20260812 14:18:19.044648 17864 raft_consensus.cc:1261] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Refusing update from remote peer aa0b5916c8ad44a191e11342b26c076d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:19.044755 17539 raft_consensus.cc:1261] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb [term 1 FOLLOWER]: Refusing update from remote peer aa0b5916c8ad44a191e11342b26c076d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:19.044910 17995 consensus_queue.cc:1035] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [LEADER]: Connected to new peer: Peer: permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:19.045199 17995 consensus_queue.cc:1035] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [LEADER]: Connected to new peer: Peer: permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:19.053789 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:19.051018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:45889 (local address 127.8.85.2:37335)
0812 14:18:19.051058 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:19.051060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:19.051088 (+ 28us) server_negotiation.cc:407] Connection header received
0812 14:18:19.051108 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:19.051109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:19.051119 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:19.051172 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:18:19.051246 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.051958 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.052994 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.053307 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.053372 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:19.053578 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:18:19.053605 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:18:19.053650 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:19.053727 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 14:18:19.053744 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:18:19.054018 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:19.050818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:37335 (local address 127.8.85.1:45889)
0812 14:18:19.050991 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:19.051004 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:19.051056 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:19.051174 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:19.051174 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:19.051176 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:18:19.051210 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:19.051211 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.052027 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.052027 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:19.052918 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:19.052921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.053408 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.053409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:19.053478 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:19.053495 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:18:19.053673 (+ 178us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:18:19.053675 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:18:19.053725 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 14:18:19.053769 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260812 14:18:19.062325 17349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:19.062451 17349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:19.063087 17349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215: Sending DeleteTablet for 3 replicas of tablet 8f23ab28123b49ccb83cf8afa02718d0
I20260812 14:18:19.063371 17516 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f23ab28123b49ccb83cf8afa02718d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:19 UTC) from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:47784
I20260812 14:18:19.063460 17681 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f23ab28123b49ccb83cf8afa02718d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:19 UTC) from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:39426
I20260812 14:18:19.063500 17986 tablet_replica.cc:330] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb: stopping tablet replica
I20260812 14:18:19.063552 17988 tablet_replica.cc:330] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d: stopping tablet replica
I20260812 14:18:19.063578 17986 raft_consensus.cc:2229] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:19.063614 17986 raft_consensus.cc:2258] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:19.063616 17988 raft_consensus.cc:2229] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:18:19.063701 17988 raft_consensus.cc:2258] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:19.063748 17842 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f23ab28123b49ccb83cf8afa02718d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:19 UTC) from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:50438
I20260812 14:18:19.063848 17987 tablet_replica.cc:330] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211: stopping tablet replica
I20260812 14:18:19.063920 17987 raft_consensus.cc:2229] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:19.063973 17987 raft_consensus.cc:2258] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:19.064061 17986 ts_tablet_manager.cc:1905] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:19.065809 17986 ts_tablet_manager.cc:1918] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:18:19.065881 17986 log.cc:1196] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/wal/wals/8f23ab28123b49ccb83cf8afa02718d0
I20260812 14:18:19.066118 17986 ts_tablet_manager.cc:1939] T 8f23ab28123b49ccb83cf8afa02718d0 P 32284027752745229022fbd48077dcbb: Deleting consensus metadata
I20260812 14:18:19.066584 17987 ts_tablet_manager.cc:1905] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:19.067612 17988 ts_tablet_manager.cc:1905] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:19.068172 17987 ts_tablet_manager.cc:1918] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:18:19.068248 17987 log.cc:1196] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/8f23ab28123b49ccb83cf8afa02718d0
I20260812 14:18:19.068483 17987 ts_tablet_manager.cc:1939] T 8f23ab28123b49ccb83cf8afa02718d0 P 679da97817a24619b9fea715485a2211: Deleting consensus metadata
I20260812 14:18:19.068826 17334 catalog_manager.cc:4841] TS 679da97817a24619b9fea715485a2211 (127.8.85.3:41515): tablet 8f23ab28123b49ccb83cf8afa02718d0 (table test-table [id=fbe25ea18639475a8a1bb44672d21bb0]) successfully deleted
I20260812 14:18:19.069072 17988 ts_tablet_manager.cc:1918] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:18:19.069146 17988 log.cc:1196] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/8f23ab28123b49ccb83cf8afa02718d0
I20260812 14:18:19.069351 17988 ts_tablet_manager.cc:1939] T 8f23ab28123b49ccb83cf8afa02718d0 P aa0b5916c8ad44a191e11342b26c076d: Deleting consensus metadata
I20260812 14:18:19.069705 17335 catalog_manager.cc:4841] TS aa0b5916c8ad44a191e11342b26c076d (127.8.85.2:37335): tablet 8f23ab28123b49ccb83cf8afa02718d0 (table test-table [id=fbe25ea18639475a8a1bb44672d21bb0]) successfully deleted
I20260812 14:18:19.069895 17333 catalog_manager.cc:4841] TS 32284027752745229022fbd48077dcbb (127.8.85.1:42667): tablet 8f23ab28123b49ccb83cf8afa02718d0 (table test-table [id=fbe25ea18639475a8a1bb44672d21bb0]) successfully deleted
Aug 12 14:18:19 dist-test-slave-5k54 krb5kdc[17300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 14:18:19 dist-test-slave-5k54 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:19 dist-test-slave-5k54 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 14:18:19.102562 17412 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:19.094060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52268 (local address 127.8.85.62:33823)
0812 14:18:19.094304 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:19.094306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:19.094417 (+ 111us) server_negotiation.cc:407] Connection header received
0812 14:18:19.094517 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:19.094518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:19.094529 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:19.094561 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:19.095336 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.096110 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.097267 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.097597 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.097667 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:19.102494 (+ 4827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52268: BlockingRecv error: recv got EOF from 127.0.0.1:52268 (error 108)
Metrics: {"server-negotiator.queue_time_us":215}
Aug 12 14:18:19 dist-test-slave-5k54 krb5kdc[17300](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM
I20260812 14:18:19.116729 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:19.107713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52276 (local address 127.8.85.62:33823)
0812 14:18:19.107758 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:19.107760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:19.107898 (+ 138us) server_negotiation.cc:407] Connection header received
0812 14:18:19.107965 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:19.107966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:19.107979 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:19.108045 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:19.108632 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.109366 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.110414 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.110701 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.110783 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:19.113583 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:19.113584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:19.113585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:19.113593 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:19.115368 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:19.115817 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:19.115818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:19.115819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:19.115893 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:19.116133 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:19.116134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:19.116135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:19.116275 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:19.116388 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:19.116539 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 14:18:19.116624 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:18:19.118628 17349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:18:19.118817 17349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:18:19.120187 17516 tablet_service.cc:1467] Processing CreateTablet for tablet 093d4da1e069480db2e6e113ab7fee1d (DEFAULT_TABLE table=test-table [id=c6ce85ff637b45169757ae75879e546d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:19.120229 17842 tablet_service.cc:1467] Processing CreateTablet for tablet 093d4da1e069480db2e6e113ab7fee1d (DEFAULT_TABLE table=test-table [id=c6ce85ff637b45169757ae75879e546d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:19.120326 17516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 093d4da1e069480db2e6e113ab7fee1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:19.120371 17842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 093d4da1e069480db2e6e113ab7fee1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:19.121724 17958 tablet_bootstrap.cc:492] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb: Bootstrap starting.
I20260812 14:18:19.121791 17955 tablet_bootstrap.cc:492] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211: Bootstrap starting.
I20260812 14:18:19.122041 17958 tablet_bootstrap.cc:654] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:19.122443 17955 tablet_bootstrap.cc:654] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:19.122624 17681 tablet_service.cc:1467] Processing CreateTablet for tablet 093d4da1e069480db2e6e113ab7fee1d (DEFAULT_TABLE table=test-table [id=c6ce85ff637b45169757ae75879e546d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:19.122743 17681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 093d4da1e069480db2e6e113ab7fee1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:19.123207 17955 tablet_bootstrap.cc:492] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211: No bootstrap required, opened a new log
I20260812 14:18:19.123251 17955 ts_tablet_manager.cc:1397] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:19.123402 17955 raft_consensus.cc:348] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.123567 17955 raft_consensus.cc:374] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:19.123596 17955 raft_consensus.cc:729] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 679da97817a24619b9fea715485a2211, State: Initialized, Role: FOLLOWER
I20260812 14:18:19.123642 17955 consensus_queue.cc:260] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.123755 17955 ts_tablet_manager.cc:1428] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:19.124249 17956 tablet_bootstrap.cc:492] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d: Bootstrap starting.
I20260812 14:18:19.124673 17956 tablet_bootstrap.cc:654] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d: Neither blocks nor log segments found. Creating new log.
W20260812 14:18:19.125329 17926 tablet.cc:2369] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:18:19.125576 17956 tablet_bootstrap.cc:492] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d: No bootstrap required, opened a new log
I20260812 14:18:19.125624 17956 ts_tablet_manager.cc:1397] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:18:19.125756 17956 raft_consensus.cc:348] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.125818 17956 raft_consensus.cc:374] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:19.125837 17956 raft_consensus.cc:729] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0b5916c8ad44a191e11342b26c076d, State: Initialized, Role: FOLLOWER
I20260812 14:18:19.125895 17956 consensus_queue.cc:260] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.125984 17956 ts_tablet_manager.cc:1428] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:19.127122 17958 tablet_bootstrap.cc:492] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb: No bootstrap required, opened a new log
I20260812 14:18:19.127185 17958 ts_tablet_manager.cc:1397] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 14:18:19.127288 17958 raft_consensus.cc:348] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.127362 17958 raft_consensus.cc:374] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:19.127383 17958 raft_consensus.cc:729] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32284027752745229022fbd48077dcbb, State: Initialized, Role: FOLLOWER
I20260812 14:18:19.127465 17958 consensus_queue.cc:260] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.127565 17958 ts_tablet_manager.cc:1428] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:19.191980 17970 raft_consensus.cc:482] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:18:19.192075 17970 raft_consensus.cc:504] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.192269 17970 leader_election.cc:288] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa0b5916c8ad44a191e11342b26c076d (127.8.85.2:37335), 679da97817a24619b9fea715485a2211 (127.8.85.3:41515)
I20260812 14:18:19.192514 17704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "093d4da1e069480db2e6e113ab7fee1d" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0b5916c8ad44a191e11342b26c076d" is_pre_election: true
I20260812 14:18:19.192513 17864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "093d4da1e069480db2e6e113ab7fee1d" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679da97817a24619b9fea715485a2211" is_pre_election: true
I20260812 14:18:19.192622 17704 raft_consensus.cc:2454] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 0.
I20260812 14:18:19.192628 17864 raft_consensus.cc:2454] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 0.
I20260812 14:18:19.192790 17456 leader_election.cc:302] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32284027752745229022fbd48077dcbb, aa0b5916c8ad44a191e11342b26c076d; no voters:
I20260812 14:18:19.192927 17970 raft_consensus.cc:2781] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:18:19.192992 17970 raft_consensus.cc:482] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:18:19.193025 17970 raft_consensus.cc:3037] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:19.193722 17970 raft_consensus.cc:504] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.193869 17970 leader_election.cc:288] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 election: Requested vote from peers aa0b5916c8ad44a191e11342b26c076d (127.8.85.2:37335), 679da97817a24619b9fea715485a2211 (127.8.85.3:41515)
I20260812 14:18:19.194005 17704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "093d4da1e069480db2e6e113ab7fee1d" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0b5916c8ad44a191e11342b26c076d"
I20260812 14:18:19.194108 17704 raft_consensus.cc:3037] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:19.194116 17864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "093d4da1e069480db2e6e113ab7fee1d" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679da97817a24619b9fea715485a2211"
I20260812 14:18:19.194200 17864 raft_consensus.cc:3037] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:19.194875 17704 raft_consensus.cc:2454] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 1.
I20260812 14:18:19.194916 17864 raft_consensus.cc:2454] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 1.
I20260812 14:18:19.195061 17456 leader_election.cc:302] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32284027752745229022fbd48077dcbb, aa0b5916c8ad44a191e11342b26c076d; no voters:
I20260812 14:18:19.195152 17970 raft_consensus.cc:2781] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:19.195199 17970 raft_consensus.cc:686] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [term 1 LEADER]: Becoming Leader. State: Replica: 32284027752745229022fbd48077dcbb, State: Running, Role: LEADER
I20260812 14:18:19.195251 17970 consensus_queue.cc:237] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.195878 17349 catalog_manager.cc:5495] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb reported cstate change: term changed from 0 to 1, leader changed from <none> to 32284027752745229022fbd48077dcbb (127.8.85.1). New cstate: current_term: 1 leader_uuid: "32284027752745229022fbd48077dcbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } health_report { overall_health: UNKNOWN } } }
I20260812 14:18:19.204732 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:19.201840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47824 (local address 127.8.85.1:42667)
0812 14:18:19.201880 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:19.201882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:19.201891 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:18:19.201981 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:19.201981 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:19.201994 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:19.202046 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:18:19.202264 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.203027 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.203964 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.204221 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.204282 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:19.204475 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:18:19.204511 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:18:19.204580 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:19.204636 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 14:18:19.204656 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:18:19.205479 17704 raft_consensus.cc:1261] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Refusing update from remote peer 32284027752745229022fbd48077dcbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:19.205498 17864 raft_consensus.cc:1261] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Refusing update from remote peer 32284027752745229022fbd48077dcbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:19.205693 17980 consensus_queue.cc:1035] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:19.205801 17970 consensus_queue.cc:1035] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:19.208184 17349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:19.208285 17349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:19.208879 17349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215: Sending DeleteTablet for 3 replicas of tablet 093d4da1e069480db2e6e113ab7fee1d
I20260812 14:18:19.209157 17516 tablet_service.cc:1514] Processing DeleteTablet for tablet 093d4da1e069480db2e6e113ab7fee1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:19 UTC) from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:47784
I20260812 14:18:19.209164 17681 tablet_service.cc:1514] Processing DeleteTablet for tablet 093d4da1e069480db2e6e113ab7fee1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:19 UTC) from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:39426
I20260812 14:18:19.209252 17986 tablet_replica.cc:330] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb: stopping tablet replica
I20260812 14:18:19.209271 17988 tablet_replica.cc:330] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d: stopping tablet replica
I20260812 14:18:19.209324 17988 raft_consensus.cc:2229] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:19.209319 17986 raft_consensus.cc:2229] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:18:19.209368 17988 raft_consensus.cc:2258] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:19.209403 17986 raft_consensus.cc:2258] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:19.209506 17842 tablet_service.cc:1514] Processing DeleteTablet for tablet 093d4da1e069480db2e6e113ab7fee1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:19 UTC) from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:50438
I20260812 14:18:19.209607 17987 tablet_replica.cc:330] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211: stopping tablet replica
I20260812 14:18:19.209573 17349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 14:18:19.209666 17987 raft_consensus.cc:2229] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260812 14:18:19.209700 17349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:18:19.209707 17987 raft_consensus.cc:2258] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:19.209698 17988 ts_tablet_manager.cc:1905] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:19.210034 17986 ts_tablet_manager.cc:1905] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:19.210867 17842 tablet_service.cc:1467] Processing CreateTablet for tablet 20fbf823d75341f7abc45301dab1cf6c (DEFAULT_TABLE table=test-table [id=c7cd85c04f064cc3bcbd8aa6bfcac39a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:19.210923 17516 tablet_service.cc:1467] Processing CreateTablet for tablet 20fbf823d75341f7abc45301dab1cf6c (DEFAULT_TABLE table=test-table [id=c7cd85c04f064cc3bcbd8aa6bfcac39a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:19.210992 17842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20fbf823d75341f7abc45301dab1cf6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:19.211015 17516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20fbf823d75341f7abc45301dab1cf6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:19.211241 17988 ts_tablet_manager.cc:1918] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:19.211293 17988 log.cc:1196] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/093d4da1e069480db2e6e113ab7fee1d
I20260812 14:18:19.211544 17988 ts_tablet_manager.cc:1939] T 093d4da1e069480db2e6e113ab7fee1d P aa0b5916c8ad44a191e11342b26c076d: Deleting consensus metadata
I20260812 14:18:19.211640 17986 ts_tablet_manager.cc:1918] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:19.211690 17986 log.cc:1196] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-0/wal/wals/093d4da1e069480db2e6e113ab7fee1d
I20260812 14:18:19.211915 17986 ts_tablet_manager.cc:1939] T 093d4da1e069480db2e6e113ab7fee1d P 32284027752745229022fbd48077dcbb: Deleting consensus metadata
I20260812 14:18:19.212206 17681 tablet_service.cc:1467] Processing CreateTablet for tablet 20fbf823d75341f7abc45301dab1cf6c (DEFAULT_TABLE table=test-table [id=c7cd85c04f064cc3bcbd8aa6bfcac39a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:19.212272 17987 ts_tablet_manager.cc:1905] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:19.212309 17681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20fbf823d75341f7abc45301dab1cf6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:19.212347 17958 tablet_bootstrap.cc:492] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb: Bootstrap starting.
I20260812 14:18:19.212615 17958 tablet_bootstrap.cc:654] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:19.212946 17335 catalog_manager.cc:4841] TS aa0b5916c8ad44a191e11342b26c076d (127.8.85.2:37335): tablet 093d4da1e069480db2e6e113ab7fee1d (table test-table [id=c6ce85ff637b45169757ae75879e546d]) successfully deleted
I20260812 14:18:19.213227 17958 tablet_bootstrap.cc:492] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb: No bootstrap required, opened a new log
I20260812 14:18:19.213272 17958 ts_tablet_manager.cc:1397] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 14:18:19.213371 17958 raft_consensus.cc:348] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.213460 17958 raft_consensus.cc:374] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:19.213462 17956 tablet_bootstrap.cc:492] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d: Bootstrap starting.
I20260812 14:18:19.213486 17958 raft_consensus.cc:729] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32284027752745229022fbd48077dcbb, State: Initialized, Role: FOLLOWER
I20260812 14:18:19.213528 17958 consensus_queue.cc:260] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.213546 17987 ts_tablet_manager.cc:1918] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:19.213595 17987 log.cc:1196] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/093d4da1e069480db2e6e113ab7fee1d
I20260812 14:18:19.213620 17958 ts_tablet_manager.cc:1428] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:19.213785 17956 tablet_bootstrap.cc:654] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:19.213810 17987 ts_tablet_manager.cc:1939] T 093d4da1e069480db2e6e113ab7fee1d P 679da97817a24619b9fea715485a2211: Deleting consensus metadata
I20260812 14:18:19.213888 17333 catalog_manager.cc:4841] TS 32284027752745229022fbd48077dcbb (127.8.85.1:42667): tablet 093d4da1e069480db2e6e113ab7fee1d (table test-table [id=c6ce85ff637b45169757ae75879e546d]) successfully deleted
I20260812 14:18:19.213968 17955 tablet_bootstrap.cc:492] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211: Bootstrap starting.
I20260812 14:18:19.214215 17334 catalog_manager.cc:4841] TS 679da97817a24619b9fea715485a2211 (127.8.85.3:41515): tablet 093d4da1e069480db2e6e113ab7fee1d (table test-table [id=c6ce85ff637b45169757ae75879e546d]) successfully deleted
I20260812 14:18:19.214354 17955 tablet_bootstrap.cc:654] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:19.214902 17955 tablet_bootstrap.cc:492] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211: No bootstrap required, opened a new log
I20260812 14:18:19.214977 17955 ts_tablet_manager.cc:1397] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:18:19.215093 17956 tablet_bootstrap.cc:492] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d: No bootstrap required, opened a new log
I20260812 14:18:19.215142 17956 ts_tablet_manager.cc:1397] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 14:18:19.215133 17955 raft_consensus.cc:348] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.215210 17955 raft_consensus.cc:374] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:19.215237 17955 raft_consensus.cc:729] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 679da97817a24619b9fea715485a2211, State: Initialized, Role: FOLLOWER
I20260812 14:18:19.215250 17956 raft_consensus.cc:348] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.215286 17955 consensus_queue.cc:260] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.215320 17956 raft_consensus.cc:374] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:19.215344 17956 raft_consensus.cc:729] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa0b5916c8ad44a191e11342b26c076d, State: Initialized, Role: FOLLOWER
I20260812 14:18:19.215402 17955 ts_tablet_manager.cc:1428] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:19.215410 17956 consensus_queue.cc:260] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.215526 17956 ts_tablet_manager.cc:1428] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:19.589277 17970 raft_consensus.cc:482] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:18:19.589385 17970 raft_consensus.cc:504] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.589579 17970 leader_election.cc:288] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa0b5916c8ad44a191e11342b26c076d (127.8.85.2:37335), 679da97817a24619b9fea715485a2211 (127.8.85.3:41515)
I20260812 14:18:19.589828 17704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20fbf823d75341f7abc45301dab1cf6c" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0b5916c8ad44a191e11342b26c076d" is_pre_election: true
I20260812 14:18:19.589859 17864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20fbf823d75341f7abc45301dab1cf6c" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679da97817a24619b9fea715485a2211" is_pre_election: true
I20260812 14:18:19.589959 17864 raft_consensus.cc:2454] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 0.
I20260812 14:18:19.589965 17704 raft_consensus.cc:2454] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 0.
I20260812 14:18:19.590130 17455 leader_election.cc:302] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32284027752745229022fbd48077dcbb, 679da97817a24619b9fea715485a2211; no voters:
I20260812 14:18:19.590268 17970 raft_consensus.cc:2781] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:18:19.590318 17970 raft_consensus.cc:482] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:18:19.590337 17970 raft_consensus.cc:3037] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:19.591087 17970 raft_consensus.cc:504] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.591231 17970 leader_election.cc:288] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 election: Requested vote from peers aa0b5916c8ad44a191e11342b26c076d (127.8.85.2:37335), 679da97817a24619b9fea715485a2211 (127.8.85.3:41515)
I20260812 14:18:19.591403 17704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20fbf823d75341f7abc45301dab1cf6c" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa0b5916c8ad44a191e11342b26c076d"
I20260812 14:18:19.591418 17864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20fbf823d75341f7abc45301dab1cf6c" candidate_uuid: "32284027752745229022fbd48077dcbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "679da97817a24619b9fea715485a2211"
I20260812 14:18:19.591502 17704 raft_consensus.cc:3037] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:19.591511 17864 raft_consensus.cc:3037] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:19.592257 17864 raft_consensus.cc:2454] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 1.
I20260812 14:18:19.592257 17704 raft_consensus.cc:2454] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32284027752745229022fbd48077dcbb in term 1.
I20260812 14:18:19.592434 17456 leader_election.cc:302] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32284027752745229022fbd48077dcbb, aa0b5916c8ad44a191e11342b26c076d; no voters:
I20260812 14:18:19.592540 17970 raft_consensus.cc:2781] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:19.592590 17970 raft_consensus.cc:686] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [term 1 LEADER]: Becoming Leader. State: Replica: 32284027752745229022fbd48077dcbb, State: Running, Role: LEADER
I20260812 14:18:19.592645 17970 consensus_queue.cc:237] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } }
I20260812 14:18:19.593303 17348 catalog_manager.cc:5495] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb reported cstate change: term changed from 0 to 1, leader changed from <none> to 32284027752745229022fbd48077dcbb (127.8.85.1). New cstate: current_term: 1 leader_uuid: "32284027752745229022fbd48077dcbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32284027752745229022fbd48077dcbb" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 } health_report { overall_health: UNKNOWN } } }
I20260812 14:18:19.659296 17952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:19.657044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:37873 (local address 127.8.85.1:42667)
0812 14:18:19.657114 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:19.657115 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:19.657156 (+ 41us) server_negotiation.cc:407] Connection header received
0812 14:18:19.657228 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:19.657229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:19.657239 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:19.657287 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:18:19.657396 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.657862 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.658614 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.658803 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.658875 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:19.659016 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:18:19.659054 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:18:19.659115 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:19.659167 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 14:18:19.659204 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":49}
I20260812 14:18:19.659299 18056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:19.656969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:42667 (local address 127.8.85.1:37873)
0812 14:18:19.657094 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:19.657107 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:19.657172 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:19.657297 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:19.657298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:19.657299 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:18:19.657348 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:19.657349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.657944 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.657945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:19.658524 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:19.658525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:19.658886 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:19.658887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:19.658937 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:19.658941 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:18:19.659116 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:18:19.659119 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:18:19.659166 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 14:18:19.659225 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":3,"thread_start_us":67,"threads_started":1}
I20260812 14:18:19.661263 17704 raft_consensus.cc:1261] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Refusing update from remote peer 32284027752745229022fbd48077dcbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:19.661476 17864 raft_consensus.cc:1261] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Refusing update from remote peer 32284027752745229022fbd48077dcbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:19.661527 17980 consensus_queue.cc:1035] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa0b5916c8ad44a191e11342b26c076d" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 37335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:19.661680 17966 consensus_queue.cc:1035] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "679da97817a24619b9fea715485a2211" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:19.672027 17348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:19.672113 17348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:19.672884 17348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9ccb7b96b5a4438ae0bd40d6962e215: Sending DeleteTablet for 3 replicas of tablet 20fbf823d75341f7abc45301dab1cf6c
I20260812 14:18:19.673142 17681 tablet_service.cc:1514] Processing DeleteTablet for tablet 20fbf823d75341f7abc45301dab1cf6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:19 UTC) from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:39426
I20260812 14:18:19.673151 17516 tablet_service.cc:1514] Processing DeleteTablet for tablet 20fbf823d75341f7abc45301dab1cf6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:19 UTC) from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:47784
I20260812 14:18:19.673249 17986 tablet_replica.cc:330] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb: stopping tablet replica
I20260812 14:18:19.673250 17988 tablet_replica.cc:330] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d: stopping tablet replica
I20260812 14:18:19.673218 17842 tablet_service.cc:1514] Processing DeleteTablet for tablet 20fbf823d75341f7abc45301dab1cf6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:19 UTC) from {username='oryx', principal='oryx/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:50438
I20260812 14:18:19.673302 17988 raft_consensus.cc:2229] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:19.673298 17986 raft_consensus.cc:2229] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:18:19.673342 17988 raft_consensus.cc:2258] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:19.673382 17986 raft_consensus.cc:2258] T 20fbf823d75341f7abc45301dab1cf6c P 32284027752745229022fbd48077dcbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:19.673454 17987 tablet_replica.cc:330] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211: stopping tablet replica
I20260812 14:18:19.673519 17987 raft_consensus.cc:2229] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:19.673563 17987 raft_consensus.cc:2258] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:19.673588 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 17436
I20260812 14:18:19.673774 17988 ts_tablet_manager.cc:1905] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:19.674407 17987 ts_tablet_manager.cc:1905] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:19.675621 17988 ts_tablet_manager.cc:1918] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:18:19.675678 17988 log.cc:1196] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/20fbf823d75341f7abc45301dab1cf6c
I20260812 14:18:19.676038 17988 ts_tablet_manager.cc:1939] T 20fbf823d75341f7abc45301dab1cf6c P aa0b5916c8ad44a191e11342b26c076d: Deleting consensus metadata
I20260812 14:18:19.676432 17335 catalog_manager.cc:4841] TS aa0b5916c8ad44a191e11342b26c076d (127.8.85.2:37335): tablet 20fbf823d75341f7abc45301dab1cf6c (table test-table [id=c7cd85c04f064cc3bcbd8aa6bfcac39a]) successfully deleted
I20260812 14:18:19.676672 17987 ts_tablet_manager.cc:1918] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:18:19.676743 17987 log.cc:1196] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/20fbf823d75341f7abc45301dab1cf6c
I20260812 14:18:19.676954 17987 ts_tablet_manager.cc:1939] T 20fbf823d75341f7abc45301dab1cf6c P 679da97817a24619b9fea715485a2211: Deleting consensus metadata
I20260812 14:18:19.677371 17334 catalog_manager.cc:4841] TS 679da97817a24619b9fea715485a2211 (127.8.85.3:41515): tablet 20fbf823d75341f7abc45301dab1cf6c (table test-table [id=c7cd85c04f064cc3bcbd8aa6bfcac39a]) successfully deleted
W20260812 14:18:19.685786 17333 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:42667 (error 108)
I20260812 14:18:19.686409 18061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:19.686237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:42667 (local address 127.0.0.1:47838)
0812 14:18:19.686335 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:19.686372 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:42667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":48,"threads_started":1}
I20260812 14:18:19.686475 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 17602
W20260812 14:18:19.686563 17333 catalog_manager.cc:4584] TS 32284027752745229022fbd48077dcbb (127.8.85.1:42667): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 20fbf823d75341f7abc45301dab1cf6c: Network error: Client connection negotiation failed: client connection to 127.8.85.1:42667: connect: Connection refused (error 111)
I20260812 14:18:19.694703 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 17769
I20260812 14:18:19.701727 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 17315
2026-08-12T14:18:19Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3035 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:18:19 dist-test-slave-5k54 krb5kdc[18065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:18:19 dist-test-slave-5k54 krb5kdc[18065](info): set up 2 sockets
Aug 12 14:18:19 dist-test-slave-5k54 krb5kdc[18065](info): commencing operation
krb5kdc: starting...
W20260812 14:18:20.823320 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:18:20 dist-test-slave-5k54 krb5kdc[18065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:18:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:18:20.950413 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:39791
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:44641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:39791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:21.032534 18083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:21.032745 18083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:21.032771 18083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:21.033977 18083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:21.034022 18083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:21.034036 18083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:21.034075 18083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:21.034107 18083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:21.035656 18083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:44641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:39791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:39791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:21.035900 18083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:21.036154 18083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:21.041162 18091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:21.041134 18089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:21.041177 18088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:21.041330 18083 server_base.cc:1034] running on GCE node
I20260812 14:18:21.041550 18083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:21.041801 18083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:21.042977 18083 hybrid_clock.cc:648] HybridClock initialized: now 1786544301042950 us; error 43 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T14:18:21Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1364 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:18:21 dist-test-slave-5k54 krb5kdc[18096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:18:21 dist-test-slave-5k54 krb5kdc[18096](info): set up 2 sockets
Aug 12 14:18:21 dist-test-slave-5k54 krb5kdc[18096](info): commencing operation
krb5kdc: starting...
W20260812 14:18:22.195606 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:18:22 dist-test-slave-5k54 krb5kdc[18096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:18:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:18:22.316082 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:45087
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:35739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:45087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:22.397985 18109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:22.398183 18109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:22.398205 18109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:22.399291 18109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:22.399325 18109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:22.399340 18109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:22.399353 18109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:22.399364 18109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:22.400791 18109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:35739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:45087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:45087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:22.400983 18109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:22.401146 18109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:22.405848 18117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:22.405833 18115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:22.405834 18119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:22.406230 18109 server_base.cc:1034] running on GCE node
I20260812 14:18:22.406428 18109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:22.406657 18109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:22.407796 18109 hybrid_clock.cc:648] HybridClock initialized: now 1786544302407783 us; error 27 us; skew 500 ppm
Aug 12 14:18:22 dist-test-slave-5k54 krb5kdc[18096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544302, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:22.410223 18109 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:18:22.410687 18109 webserver.cc:460] Webserver started at http://127.8.85.62:41615/ using document root <none> and password file <none>
I20260812 14:18:22.410917 18109 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:22.410987 18109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:22.411142 18109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:22.412281 18109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "3e89fe097ea34571bb978f00688188da"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "2e2d5b37c5dd474a47f1e885a5899e73"
server_key_iv: "54f99b63f6c97a89d195107d9f92faad"
server_key_version: "encryptionkey@0"
I20260812 14:18:22.412652 18109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "3e89fe097ea34571bb978f00688188da"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "2e2d5b37c5dd474a47f1e885a5899e73"
server_key_iv: "54f99b63f6c97a89d195107d9f92faad"
server_key_version: "encryptionkey@0"
I20260812 14:18:22.413981 18109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:18:22.414686 18128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:22.414852 18109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:22.414979 18109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "3e89fe097ea34571bb978f00688188da"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "2e2d5b37c5dd474a47f1e885a5899e73"
server_key_iv: "54f99b63f6c97a89d195107d9f92faad"
server_key_version: "encryptionkey@0"
I20260812 14:18:22.415059 18109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:22.431599 18109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:22.433504 18109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:22.433669 18109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:22.437742 18109 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:45087
I20260812 14:18:22.437783 18189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:45087 every 8 connection(s)
I20260812 14:18:22.438162 18109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:22.439025 18190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:22.440701 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 18109
I20260812 14:18:22.440827 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:22.441048 8532 external_mini_cluster.cc:1428] Setting key 0407711deff76d606ddbc2af8fa3b459
I20260812 14:18:22.441687 18190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da: Bootstrap starting.
I20260812 14:18:22.442288 18190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:22.442675 18190 log.cc:824] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:22.443471 18190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da: No bootstrap required, opened a new log
I20260812 14:18:22.445678 18190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e89fe097ea34571bb978f00688188da" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45087 } }
I20260812 14:18:22.445814 18190 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:22.445837 18190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e89fe097ea34571bb978f00688188da, State: Initialized, Role: FOLLOWER
I20260812 14:18:22.446004 18190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e89fe097ea34571bb978f00688188da" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45087 } }
I20260812 14:18:22.446108 18190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:22.446148 18190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:22.446211 18190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:22.446923 18190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e89fe097ea34571bb978f00688188da" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45087 } }
I20260812 14:18:22.447047 18190 leader_election.cc:302] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e89fe097ea34571bb978f00688188da; no voters:
I20260812 14:18:22.447233 18190 leader_election.cc:288] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:22.447268 18198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:22.447464 18198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 1 LEADER]: Becoming Leader. State: Replica: 3e89fe097ea34571bb978f00688188da, State: Running, Role: LEADER
I20260812 14:18:22.447520 18190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:22.447562 18198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e89fe097ea34571bb978f00688188da" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45087 } }
I20260812 14:18:22.447885 18199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e89fe097ea34571bb978f00688188da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e89fe097ea34571bb978f00688188da" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45087 } } }
I20260812 14:18:22.447863 18198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e89fe097ea34571bb978f00688188da. Latest consensus state: current_term: 1 leader_uuid: "3e89fe097ea34571bb978f00688188da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e89fe097ea34571bb978f00688188da" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45087 } } }
I20260812 14:18:22.447965 18199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:22.447976 18198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [sys.catalog]: This master's current role is: LEADER
Aug 12 14:18:22 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:22.448634 18204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:22.449190 18204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:22.451143 18204 catalog_manager.cc:1347] Generated new cluster ID: 14af4cc649b84651ab2d74e84bcb9198
I20260812 14:18:22.451200 18204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:18:22.452656 18193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:22.441610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53262 (local address 127.8.85.62:45087)
0812 14:18:22.441802 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:22.441813 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:22.441852 (+ 39us) server_negotiation.cc:407] Connection header received
0812 14:18:22.442284 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:22.442290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:22.442336 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:22.442500 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:22.443299 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:22.444275 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:22.445417 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:22.445797 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:22.445926 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:22.448965 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:22.448970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:22.448970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:22.448982 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:22.450906 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:22.451328 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:22.451331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:22.451331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:22.451471 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:22.451678 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:22.451679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:22.451679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:22.451982 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:22.452151 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:22.452281 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 14:18:22.452426 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
I20260812 14:18:22.456835 18204 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:22.457309 18204 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:22.460860 18204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da: Generated new TSK 0
I20260812 14:18:22.461036 18204 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:18:22.492182 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:45087
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:35739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:22.595495 18223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:22.595697 18223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:22.595719 18223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:22.597011 18223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:22.597067 18223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:22.597097 18223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:18:22.598850 18223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:35739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:45087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:22.599113 18223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:22.599359 18223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:22.605317 18230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:22.605357 18229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:22.605368 18223 server_base.cc:1034] running on GCE node
W20260812 14:18:22.605540 18233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:22.605849 18223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:22.606096 18223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:22.607311 18223 hybrid_clock.cc:648] HybridClock initialized: now 1786544302607286 us; error 40 us; skew 500 ppm
Aug 12 14:18:22 dist-test-slave-5k54 krb5kdc[18096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544302, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:22.609987 18223 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:18:22.610361 18223 webserver.cc:460] Webserver started at http://127.8.85.1:36557/ using document root <none> and password file <none>
I20260812 14:18:22.610574 18223 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:22.610616 18223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:22.610707 18223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:22.611822 18223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "58b64023c57244e0bf165e4cbd526741"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "fea710addc8211afe29419dbd553a5c9"
server_key_iv: "7ba0fcf92c0fdb583b499a0a0f432fe7"
server_key_version: "encryptionkey@0"
I20260812 14:18:22.612133 18223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "58b64023c57244e0bf165e4cbd526741"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "fea710addc8211afe29419dbd553a5c9"
server_key_iv: "7ba0fcf92c0fdb583b499a0a0f432fe7"
server_key_version: "encryptionkey@0"
I20260812 14:18:22.613483 18223 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:22.614168 18241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:22.614347 18223 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:22.614428 18223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "58b64023c57244e0bf165e4cbd526741"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "fea710addc8211afe29419dbd553a5c9"
server_key_iv: "7ba0fcf92c0fdb583b499a0a0f432fe7"
server_key_version: "encryptionkey@0"
I20260812 14:18:22.614483 18223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:22.637939 18223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:22.640098 18223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:22.640213 18223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:22.640419 18223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:22.640766 18223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:22.640803 18223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:22.640829 18223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:22.640877 18223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:22.646651 18223 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:38693
I20260812 14:18:22.646739 18371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:38693 every 8 connection(s)
I20260812 14:18:22.647087 18223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:18:22.648135 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 18223
I20260812 14:18:22.648264 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:18:22.648486 8532 external_mini_cluster.cc:1428] Setting key d48d3a87f6a83b85c8be33f1ff798fe3
Aug 12 14:18:22 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544302, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:22.658038 18193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:22.647815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:59013 (local address 127.8.85.62:45087)
0812 14:18:22.647863 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:22.647865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:22.648120 (+ 255us) server_negotiation.cc:407] Connection header received
0812 14:18:22.648823 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:22.648824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:22.648870 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:22.648925 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:22.649955 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:22.650732 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:22.652191 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:22.652399 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:22.652456 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:22.655019 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:22.655020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:22.655021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:22.655028 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:22.656493 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:22.657011 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:22.657011 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:22.657012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:22.657063 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:22.657381 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:22.657382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:22.657382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:22.657494 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:22.657545 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:22.657895 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 14:18:22.657973 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:18:22.658335 18374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:22.647795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:59013)
0812 14:18:22.647976 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:22.648025 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:22.648668 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:22.649004 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:22.649007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:22.649668 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:22.649862 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:22.649867 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:22.650826 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:22.650827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:22.652080 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:22.652084 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:22.652504 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:22.652505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:22.652646 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:22.653146 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:22.653157 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:22.654907 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:22.656581 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:22.656590 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:22.656593 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:22.656924 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:22.657139 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:22.657140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:22.657140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:22.657283 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:22.657563 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:22.657570 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:22.657742 (+ 172us) client_negotiation.cc:769] Sending connection context
0812 14:18:22.657869 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 14:18:22.658051 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
I20260812 14:18:22.658905 18372 heartbeater.cc:347] Connected to a master server at 127.8.85.62:45087
I20260812 14:18:22.659015 18372 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:22.659183 18372 heartbeater.cc:511] Master 127.8.85.62:45087 requested a full tablet report, sending...
I20260812 14:18:22.659718 18151 ts_manager.cc:194] Registered new tserver with Master: 58b64023c57244e0bf165e4cbd526741 (127.8.85.1:38693)
I20260812 14:18:22.660408 18151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:59013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:18:22.678282 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:45087
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:35739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:22.761946 18377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:22.762177 18377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:22.762213 18377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:22.763384 18377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:22.763487 18377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:22.763556 18377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:18:22.765071 18377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:35739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:45087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:22.765337 18377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:22.765585 18377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:22.770665 18388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:22.770641 18385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:22.770642 18384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:22.771229 18377 server_base.cc:1034] running on GCE node
I20260812 14:18:22.771524 18377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:22.771781 18377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:22.772918 18377 hybrid_clock.cc:648] HybridClock initialized: now 1786544302772898 us; error 40 us; skew 500 ppm
Aug 12 14:18:22 dist-test-slave-5k54 krb5kdc[18096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544302, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:22.775605 18377 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:18:22.776073 18377 webserver.cc:460] Webserver started at http://127.8.85.2:36687/ using document root <none> and password file <none>
I20260812 14:18:22.776300 18377 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:22.776347 18377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:22.776521 18377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:22.777559 18377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "82dfb1e2ee334ca98828cbb6bf2ddd39"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "880ec3f32654b6c871480fce3f38eef5"
server_key_iv: "ff7468c9b7abba56d4128d70d1af5e79"
server_key_version: "encryptionkey@0"
I20260812 14:18:22.777926 18377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "82dfb1e2ee334ca98828cbb6bf2ddd39"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "880ec3f32654b6c871480fce3f38eef5"
server_key_iv: "ff7468c9b7abba56d4128d70d1af5e79"
server_key_version: "encryptionkey@0"
I20260812 14:18:22.779299 18377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:18:22.780146 18398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:22.780375 18377 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:22.780449 18377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "82dfb1e2ee334ca98828cbb6bf2ddd39"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "880ec3f32654b6c871480fce3f38eef5"
server_key_iv: "ff7468c9b7abba56d4128d70d1af5e79"
server_key_version: "encryptionkey@0"
I20260812 14:18:22.780539 18377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:22.794652 18377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:22.797113 18377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:22.797286 18377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:22.797530 18377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:22.797863 18377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:22.797920 18377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:22.797961 18377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:22.797989 18377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:22.804569 18377 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:37045
I20260812 14:18:22.804623 18529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:37045 every 8 connection(s)
I20260812 14:18:22.804967 18377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:18:22 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544302, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:22.814150 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 18377
I20260812 14:18:22.814276 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:18:22.814514 8532 external_mini_cluster.cc:1428] Setting key a224e9d90c7e9ce25b6225e41512c4df
I20260812 14:18:22.814528 18193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:22.805691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:48367 (local address 127.8.85.62:45087)
0812 14:18:22.805762 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:22.805765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:22.805990 (+ 225us) server_negotiation.cc:407] Connection header received
0812 14:18:22.806481 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:22.806482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:22.806492 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:22.806547 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:22.807801 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:22.808317 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:22.809327 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:22.809503 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:22.809551 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:22.811691 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:22.811692 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:22.811693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:22.811700 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:22.813098 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:22.813541 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:22.813543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:22.813543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:22.813601 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:22.813849 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:22.813850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:22.813851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:22.813969 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:22.814043 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:22.814333 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 14:18:22.814438 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:18:22.814718 18534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:22.805680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:48367)
0812 14:18:22.805860 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:22.805898 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:22.806389 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:22.806614 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:22.806617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:22.807412 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:22.807701 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:22.807707 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:22.808424 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:22.808425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:22.809249 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:22.809252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:22.809574 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:22.809575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:22.809671 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:22.810111 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:22.810119 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:22.811595 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:22.813200 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:22.813207 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:22.813209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:22.813434 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:22.813670 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:22.813671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:22.813671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:22.813783 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:22.814035 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:22.814041 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:22.814179 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 14:18:22.814289 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 14:18:22.814477 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":107,"threads_started":1}
I20260812 14:18:22.815225 18530 heartbeater.cc:347] Connected to a master server at 127.8.85.62:45087
I20260812 14:18:22.815337 18530 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:22.815575 18530 heartbeater.cc:511] Master 127.8.85.62:45087 requested a full tablet report, sending...
I20260812 14:18:22.816016 18151 ts_manager.cc:194] Registered new tserver with Master: 82dfb1e2ee334ca98828cbb6bf2ddd39 (127.8.85.2:37045)
I20260812 14:18:22.816511 18151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:48367
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:18:22.842061 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:45087
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:35739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:22.928591 18538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:22.928824 18538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:22.928865 18538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:22.930059 18538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:22.930126 18538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:22.930172 18538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:18:22.931814 18538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:35739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:45087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:22.932072 18538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:22.932307 18538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:22.937249 18544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:22.937409 18538 server_base.cc:1034] running on GCE node
W20260812 14:18:22.937248 18546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:22.937317 18543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:22.937793 18538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:22.938128 18538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:22.939334 18538 hybrid_clock.cc:648] HybridClock initialized: now 1786544302939316 us; error 32 us; skew 500 ppm
Aug 12 14:18:22 dist-test-slave-5k54 krb5kdc[18096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544302, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:22.941831 18538 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:18:22.942270 18538 webserver.cc:460] Webserver started at http://127.8.85.3:41503/ using document root <none> and password file <none>
I20260812 14:18:22.942503 18538 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:22.942576 18538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:22.942735 18538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:22.943936 18538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "2569b68242e14defa5f1ba8d686d5345"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "3995379ae6f6b09cc9a4489cbf15ea13"
server_key_iv: "443db7f0d865114247f871b451e6e674"
server_key_version: "encryptionkey@0"
I20260812 14:18:22.944321 18538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "2569b68242e14defa5f1ba8d686d5345"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "3995379ae6f6b09cc9a4489cbf15ea13"
server_key_iv: "443db7f0d865114247f871b451e6e674"
server_key_version: "encryptionkey@0"
I20260812 14:18:22.945663 18538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:22.946404 18554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:22.946578 18538 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:22.946679 18538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "2569b68242e14defa5f1ba8d686d5345"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "3995379ae6f6b09cc9a4489cbf15ea13"
server_key_iv: "443db7f0d865114247f871b451e6e674"
server_key_version: "encryptionkey@0"
I20260812 14:18:22.946760 18538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:22.956475 18538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:22.958202 18538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:22.958360 18538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:22.958576 18538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:22.958884 18538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:22.958941 18538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:22.958981 18538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:22.959010 18538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:22.965547 18538 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:43277
I20260812 14:18:22.965582 18681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:43277 every 8 connection(s)
I20260812 14:18:22.965922 18538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:18:22.967326 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 18538
I20260812 14:18:22.967453 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:18:22.967665 8532 external_mini_cluster.cc:1428] Setting key 13bf1db0ccdc9ab6e38e62b6953fc039
Aug 12 14:18:22 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544302, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:22.975226 18193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:22.966656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:39671 (local address 127.8.85.62:45087)
0812 14:18:22.966714 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:22.966717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:22.966922 (+ 205us) server_negotiation.cc:407] Connection header received
0812 14:18:22.967653 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:22.967654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:22.967665 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:22.967745 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:22.968938 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:22.969442 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:22.970386 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:22.970614 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:22.970707 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:22.972888 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:22.972889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:22.972890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:22.972897 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:22.974079 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:22.974437 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:22.974438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:22.974438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:22.974490 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:22.974700 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:22.974701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:22.974701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:22.974800 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:22.974849 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:22.975097 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 14:18:22.975160 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:18:22.975384 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:22.966633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:39671)
0812 14:18:22.966808 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:22.966842 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:22.967534 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:22.967822 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:22.967827 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:22.968647 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:22.968856 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:22.968861 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:22.969519 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:22.969520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:22.970287 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:22.970289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:22.970723 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:22.970724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:22.970829 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:22.971349 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:22.971365 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:22.972802 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:22.974152 (+ 1350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:22.974158 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:22.974160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:22.974371 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:22.974549 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:22.974550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:22.974550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:22.974646 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:22.974854 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:22.974858 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:22.974982 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 14:18:22.975070 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 14:18:22.975189 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260812 14:18:22.975893 18682 heartbeater.cc:347] Connected to a master server at 127.8.85.62:45087
I20260812 14:18:22.976004 18682 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:22.976184 18682 heartbeater.cc:511] Master 127.8.85.62:45087 requested a full tablet report, sending...
I20260812 14:18:22.976593 18151 ts_manager.cc:194] Registered new tserver with Master: 2569b68242e14defa5f1ba8d686d5345 (127.8.85.3:43277)
I20260812 14:18:22.976931 18151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:39671
I20260812 14:18:22.979485 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.1.keytab.
Entry for principal oryx/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.1.keytab.
WARNING: no policy specified for oryx/127.8.85.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.2.keytab.
Entry for principal oryx/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.2.keytab.
WARNING: no policy specified for oryx/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.3.keytab.
Entry for principal oryx/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.3.keytab.
I20260812 14:18:23.054762 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 18223
I20260812 14:18:23.061386 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 18377
I20260812 14:18:23.067005 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 18538
I20260812 14:18:23.074375 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 18109
I20260812 14:18:23.081445 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:45087
--webserver_interface=127.8.85.62
--webserver_port=41615
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:35739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:45087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:23.165748 18691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:23.165944 18691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:23.165967 18691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:23.167142 18691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:23.167188 18691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:23.167202 18691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:23.167215 18691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:23.167227 18691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:23.168784 18691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:35739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:45087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:45087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:23.169054 18691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:23.169291 18691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:23.174018 18699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:23.174134 18698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:23.174041 18701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:23.174319 18691 server_base.cc:1034] running on GCE node
I20260812 14:18:23.174576 18691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:23.174857 18691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:23.176023 18691 hybrid_clock.cc:648] HybridClock initialized: now 1786544303176019 us; error 38 us; skew 500 ppm
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544303, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:23.178475 18691 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:18:23.178918 18691 webserver.cc:460] Webserver started at http://127.8.85.62:41615/ using document root <none> and password file <none>
I20260812 14:18:23.179163 18691 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:23.179247 18691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:23.180629 18691 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:23.181298 18709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:23.181469 18691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:23.181571 18691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "3e89fe097ea34571bb978f00688188da"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "2e2d5b37c5dd474a47f1e885a5899e73"
server_key_iv: "54f99b63f6c97a89d195107d9f92faad"
server_key_version: "encryptionkey@0"
I20260812 14:18:23.181937 18691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:23.192368 18691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:23.194103 18691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:23.194260 18691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:23.198418 18691 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:45087
I20260812 14:18:23.198480 18765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:45087 every 8 connection(s)
I20260812 14:18:23.198865 18691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:23.200356 18766 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:18:23.201004 18766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da: Bootstrap starting.
I20260812 14:18:23.206769 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 18691
I20260812 14:18:23.207082 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:38693
--local_ip_for_outbound_sockets=127.8.85.1
--tserver_master_addrs=127.8.85.62:45087
--webserver_port=36557
--webserver_interface=127.8.85.1
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:35739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.85.1
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:18:23.209473 18766 log.cc:824] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:23.211218 18766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:18:23.211575 18766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da: Bootstrap complete.
I20260812 14:18:23.213811 18766 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e89fe097ea34571bb978f00688188da" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45087 } }
I20260812 14:18:23.214056 18766 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e89fe097ea34571bb978f00688188da, State: Initialized, Role: FOLLOWER
I20260812 14:18:23.214250 18766 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e89fe097ea34571bb978f00688188da" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45087 } }
I20260812 14:18:23.214331 18766 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:23.214383 18766 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:23.214429 18766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:18:23.215405 18766 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e89fe097ea34571bb978f00688188da" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45087 } }
I20260812 14:18:23.215561 18766 leader_election.cc:302] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e89fe097ea34571bb978f00688188da; no voters:
I20260812 14:18:23.215780 18766 leader_election.cc:288] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 14:18:23.215816 18771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 2 FOLLOWER]: Leader election won for term 2
I20260812 14:18:23.215950 18771 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [term 2 LEADER]: Becoming Leader. State: Replica: 3e89fe097ea34571bb978f00688188da, State: Running, Role: LEADER
I20260812 14:18:23.216054 18766 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:23.216037 18771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e89fe097ea34571bb978f00688188da" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45087 } }
I20260812 14:18:23.216308 18771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e89fe097ea34571bb978f00688188da. Latest consensus state: current_term: 2 leader_uuid: "3e89fe097ea34571bb978f00688188da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e89fe097ea34571bb978f00688188da" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45087 } } }
I20260812 14:18:23.216387 18771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:23.216313 18772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3e89fe097ea34571bb978f00688188da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e89fe097ea34571bb978f00688188da" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45087 } } }
I20260812 14:18:23.216439 18772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:23.216724 18775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:23.217350 18775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:23.217695 18775 catalog_manager.cc:1259] Loaded cluster ID: 14af4cc649b84651ab2d74e84bcb9198
I20260812 14:18:23.217748 18775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:18:23.219061 18775 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:23.219244 18775 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3e89fe097ea34571bb978f00688188da: Loaded TSK: 0
I20260812 14:18:23.219486 18775 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 14:18:23.290915 18768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:23.291124 18768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:23.291148 18768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:23.292382 18768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:23.292428 18768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:23.292454 18768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:18:23.293879 18768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:35739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.1.keytab
--principal=oryx/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:38693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=36557
--tserver_master_addrs=127.8.85.62:45087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:23.294131 18768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:23.294379 18768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:23.299386 18794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:23.299414 18798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:23.299738 18768 server_base.cc:1034] running on GCE node
W20260812 14:18:23.299824 18796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:23.300050 18768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:23.300303 18768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:23.301466 18768 hybrid_clock.cc:648] HybridClock initialized: now 1786544303301441 us; error 47 us; skew 500 ppm
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544303, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:23.304246 18768 init.cc:377] Logged in from keytab as oryx/127.8.85.1@KRBTEST.COM (short username oryx)
I20260812 14:18:23.304797 18768 webserver.cc:460] Webserver started at http://127.8.85.1:36557/ using document root <none> and password file <none>
I20260812 14:18:23.305065 18768 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:23.305148 18768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:23.306963 18768 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 14:18:23.307677 18805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:23.307848 18768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:18:23.307986 18768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "58b64023c57244e0bf165e4cbd526741"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "fea710addc8211afe29419dbd553a5c9"
server_key_iv: "7ba0fcf92c0fdb583b499a0a0f432fe7"
server_key_version: "encryptionkey@0"
I20260812 14:18:23.308336 18768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:23.325533 18768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:23.328156 18768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:23.328322 18768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:23.328593 18768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:23.328924 18768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:23.328958 18768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:23.329001 18768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:23.329026 18768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:23.335284 18768 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:38693
I20260812 14:18:23.335336 18935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:38693 every 8 connection(s)
I20260812 14:18:23.335698 18768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:23.343137 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 18768
I20260812 14:18:23.343542 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:37045
--local_ip_for_outbound_sockets=127.8.85.2
--tserver_master_addrs=127.8.85.62:45087
--webserver_port=36687
--webserver_interface=127.8.85.2
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:35739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.85.2
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:23.344924 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:23.336386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:57541)
0812 14:18:23.336555 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:23.336597 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:23.337057 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:23.337536 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:23.337540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:23.338092 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:23.338274 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.338279 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.339932 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.339932 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.340719 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.340721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.341547 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.341547 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.341646 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.342188 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:23.342197 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:23.344723 (+ 2526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
W20260812 14:18:23.345134 18936 heartbeater.cc:644] Failed to heartbeat to 127.8.85.62:45087 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.85.62:45087: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
W20260812 14:18:23.345381 18941 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:23.336496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:57541 (local address 127.8.85.62:45087)
0812 14:18:23.336676 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:23.336685 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:23.336718 (+ 33us) server_negotiation.cc:407] Connection header received
0812 14:18:23.337276 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:23.337286 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:23.337344 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:23.337501 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:23.338406 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.339809 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.340810 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.341447 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.341590 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.344896 (+ 3306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:57541: BlockingRecv error: recv got EOF from 127.8.85.1:57541 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":75,"threads_started":1}
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:23.350308 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:23.345316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:52519)
0812 14:18:23.345474 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:23.345488 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:23.345543 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:23.345767 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:23.345768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:23.345966 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:23.346023 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.346025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.346936 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.346937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.347601 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.347603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.347996 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.347998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.348063 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.348503 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:23.348508 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:23.350217 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
W20260812 14:18:23.350361 18945 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:23.345489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:52519 (local address 127.8.85.62:45087)
0812 14:18:23.345646 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:23.345648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:23.345656 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:18:23.345676 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:23.345677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:23.345686 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:23.345757 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:23.346105 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.346847 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.347685 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.347884 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.347970 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.350291 (+ 2321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:52519: BlockingRecv error: recv got EOF from 127.8.85.1:52519 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":71,"threads_started":1}
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:23.354945 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:23.350810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:59653)
0812 14:18:23.350895 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:23.350906 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:23.350956 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:23.351076 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:23.351076 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:23.351206 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:23.351247 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.351248 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.351937 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.351938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.352427 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.352429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.352803 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.352804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.352871 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.353287 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:23.353292 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:23.354862 (+ 1570us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260812 14:18:23.354995 18945 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:23.350850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:59653 (local address 127.8.85.62:45087)
0812 14:18:23.350887 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:23.350890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:23.350955 (+ 65us) server_negotiation.cc:407] Connection header received
0812 14:18:23.351015 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:23.351016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:23.351026 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:23.351074 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:23.351321 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.351850 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.352510 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.352716 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.352801 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.354935 (+ 2134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:59653: BlockingRecv error: recv got EOF from 127.8.85.1:59653 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 14:18:23.355285 18936 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:18:23.428915 18943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:23.429162 18943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:23.429203 18943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:23.430368 18943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:23.430433 18943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:23.430487 18943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:18:23.431993 18943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:35739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.2.keytab
--principal=oryx/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:37045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=36687
--tserver_master_addrs=127.8.85.62:45087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:23.432272 18943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:23.432510 18943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:23.437525 18952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:23.437427 18951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:23.437427 18955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:23.437774 18943 server_base.cc:1034] running on GCE node
I20260812 14:18:23.437994 18943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:23.438186 18943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:23.439349 18943 hybrid_clock.cc:648] HybridClock initialized: now 1786544303439336 us; error 27 us; skew 500 ppm
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544303, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:23.441792 18943 init.cc:377] Logged in from keytab as oryx/127.8.85.2@KRBTEST.COM (short username oryx)
I20260812 14:18:23.442226 18943 webserver.cc:460] Webserver started at http://127.8.85.2:36687/ using document root <none> and password file <none>
I20260812 14:18:23.442456 18943 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:23.442540 18943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:23.443986 18943 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:23.444641 18965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:23.444886 18943 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:23.444988 18943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "82dfb1e2ee334ca98828cbb6bf2ddd39"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "880ec3f32654b6c871480fce3f38eef5"
server_key_iv: "ff7468c9b7abba56d4128d70d1af5e79"
server_key_version: "encryptionkey@0"
I20260812 14:18:23.445335 18943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:23.456678 18943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:23.458520 18943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:23.458628 18943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:23.458792 18943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:23.459102 18943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:23.459137 18943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:23.459159 18943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:23.459204 18943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:23.465312 18943 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:37045
I20260812 14:18:23.465379 19096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:37045 every 8 connection(s)
I20260812 14:18:23.465735 18943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:18:23.470485 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 18943
I20260812 14:18:23.470834 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:43277
--local_ip_for_outbound_sockets=127.8.85.3
--tserver_master_addrs=127.8.85.62:45087
--webserver_port=41503
--webserver_interface=127.8.85.3
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:35739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.85.3
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:23.473469 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:23.466465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:59633)
0812 14:18:23.466639 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:23.466675 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:23.467145 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:23.467542 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:23.467546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:23.468114 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:23.468300 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.468304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.468954 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.468954 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.469754 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.469756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.470081 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.470081 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.470178 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.470697 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:23.470709 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:23.473317 (+ 2608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
W20260812 14:18:23.473521 18945 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:23.466497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:59633 (local address 127.8.85.62:45087)
0812 14:18:23.466577 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:23.466579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:23.466760 (+ 181us) server_negotiation.cc:407] Connection header received
0812 14:18:23.467323 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:23.467324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:23.467339 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:23.467400 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:23.468379 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.468873 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.469826 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.470007 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.470059 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.473429 (+ 3370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:59633: BlockingRecv error: recv got EOF from 127.8.85.2:59633 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 14:18:23.473765 19097 heartbeater.cc:644] Failed to heartbeat to 127.8.85.62:45087 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.85.62:45087: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:23.478781 19101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:23.474032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:45371)
0812 14:18:23.474174 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:23.474184 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:23.474237 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:23.474356 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:23.474357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:23.474484 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:23.474525 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.474526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.475408 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.475409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.475979 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.475981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.476502 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.476503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.476545 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.476996 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:23.477000 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:23.478726 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
W20260812 14:18:23.478925 18945 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:23.474089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:45371 (local address 127.8.85.62:45087)
0812 14:18:23.474151 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:23.474154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:23.474244 (+ 90us) server_negotiation.cc:407] Connection header received
0812 14:18:23.474274 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:23.474275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:23.474289 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:23.474353 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:23.474592 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.475324 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.476060 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.476402 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.476477 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.478844 (+ 2367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:45371: BlockingRecv error: recv got EOF from 127.8.85.2:45371 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:23.483351 19101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:23.479036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:42883)
0812 14:18:23.479071 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:23.479083 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:23.479128 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:23.479321 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:23.479322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:23.479501 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:23.479554 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.479556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.480162 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.480163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.480660 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.480662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.481029 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.481029 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.481098 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.481528 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:23.481532 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:23.483275 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":18}
W20260812 14:18:23.483405 18945 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:23.479144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:42883 (local address 127.8.85.62:45087)
0812 14:18:23.479202 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:23.479204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:23.479217 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:18:23.479236 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:23.479237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:23.479244 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:23.479302 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:23.479636 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.480083 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.480757 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.480947 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.481017 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.483347 (+ 2330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:42883: BlockingRecv error: recv got EOF from 127.8.85.2:42883 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260812 14:18:23.483521 19097 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:18:23.557677 19100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:23.557868 19100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:23.557894 19100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:23.559044 19100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:23.559087 19100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:23.559113 19100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:18:23.560621 19100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:35739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/krb5kdc/oryx_127.8.85.3.keytab
--principal=oryx/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:43277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=41503
--tserver_master_addrs=127.8.85.62:45087
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:23.560864 19100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:23.561100 19100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:23.566181 19108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:23.566198 19109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:23.566179 19111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:23.566452 19100 server_base.cc:1034] running on GCE node
I20260812 14:18:23.566700 19100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:23.566938 19100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:23.568096 19100 hybrid_clock.cc:648] HybridClock initialized: now 1786544303568083 us; error 24 us; skew 500 ppm
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544303, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:23.570587 19100 init.cc:377] Logged in from keytab as oryx/127.8.85.3@KRBTEST.COM (short username oryx)
I20260812 14:18:23.571089 19100 webserver.cc:460] Webserver started at http://127.8.85.3:41503/ using document root <none> and password file <none>
I20260812 14:18:23.571333 19100 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:23.571424 19100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:23.572784 19100 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:18:23.573354 19119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:23.573529 19100 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:23.573640 19100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "2569b68242e14defa5f1ba8d686d5345"
format_stamp: "Formatted at 2026-08-12 14:18:22 on dist-test-slave-5k54"
server_key: "3995379ae6f6b09cc9a4489cbf15ea13"
server_key_iv: "443db7f0d865114247f871b451e6e674"
server_key_version: "encryptionkey@0"
I20260812 14:18:23.573985 19100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:23.604732 19100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:23.607208 19100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:23.607384 19100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:23.607671 19100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:23.608006 19100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:23.608060 19100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:23.608095 19100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:23.608140 19100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:23.614353 19100 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:43277
I20260812 14:18:23.614418 19245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:43277 every 8 connection(s)
I20260812 14:18:23.614749 19100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:18:23.618062 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 19100
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 14:18:23.623000 18945 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:23.615489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:35921 (local address 127.8.85.62:45087)
0812 14:18:23.615564 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:23.615566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:23.615768 (+ 202us) server_negotiation.cc:407] Connection header received
0812 14:18:23.616419 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:23.616421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:23.616442 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:23.616496 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:23.617554 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.618229 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.619592 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.619778 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.619831 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.622926 (+ 3095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:35921: BlockingRecv error: recv got EOF from 127.8.85.3:35921 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:18:23.623037 19248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:23.615474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:35921)
0812 14:18:23.615649 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:23.615686 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:23.616298 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:23.616567 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:23.616571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:23.617207 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:23.617460 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.617465 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.618343 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.618344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.619505 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.619509 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.619867 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.619868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.620016 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.620541 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:23.620549 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:23.622825 (+ 2276us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
W20260812 14:18:23.623265 19246 heartbeater.cc:644] Failed to heartbeat to 127.8.85.62:45087 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.85.62:45087: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
I20260812 14:18:23.625891 18941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:23.618601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53274 (local address 127.8.85.62:45087)
0812 14:18:23.618667 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:23.618670 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:23.618763 (+ 93us) server_negotiation.cc:407] Connection header received
0812 14:18:23.618793 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:23.618794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:23.618807 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:23.618891 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:23.619471 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.620099 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.620847 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.621169 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.621241 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.622512 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:23.622516 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:23.622517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:23.622528 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:23.624538 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:23.624947 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:23.624950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:23.624951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:23.625041 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:23.625247 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:23.625247 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:23.625247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:23.625450 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:23.625594 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:23.625691 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 14:18:23.625794 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:23.628175 19248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:23.623521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:44289)
0812 14:18:23.623563 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:23.623573 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:23.623621 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:23.623750 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:23.623750 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:23.623856 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:23.623899 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.623900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.624736 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.624737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.625448 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.625450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.625865 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.625866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.625954 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.626348 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:23.626353 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:23.628134 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 14:18:23.628269 18945 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:23.623543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:44289 (local address 127.8.85.62:45087)
0812 14:18:23.623580 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:23.623582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:23.623607 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:18:23.623671 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:23.623672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:23.623682 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:23.623748 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:23.623971 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.624663 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.625523 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.625788 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.625850 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.628209 (+ 2359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:44289: BlockingRecv error: recv got EOF from 127.8.85.3:44289 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:23 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:23.632246 19248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:23.628443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:47453)
0812 14:18:23.628541 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:23.628552 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:23.628600 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:23.628723 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:23.628723 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:23.628847 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:23.628870 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.628871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.629454 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.629455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.629933 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:23.629935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.630279 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.630280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:23.630332 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.630658 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:23.630662 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:23.632171 (+ 1509us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":82}
W20260812 14:18:23.632288 18945 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:23.628500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:47453 (local address 127.8.85.62:45087)
0812 14:18:23.628532 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:23.628534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:23.628585 (+ 51us) server_negotiation.cc:407] Connection header received
0812 14:18:23.628656 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:23.628657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:23.628665 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:23.628718 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:23.628948 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.629374 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.630029 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:23.630208 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:23.630279 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:23.632234 (+ 1955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:47453: BlockingRecv error: recv got EOF from 127.8.85.3:47453 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260812 14:18:23.632470 19246 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 14:18:24 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:24 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:24.360464 19251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:24.355773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:50685)
0812 14:18:24.355904 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:24.355918 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:24.355972 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:24.356139 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:24.356140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:24.356308 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:24.356358 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:24.356360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:24.356984 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:24.356985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:24.357496 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:24.357497 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:24.357808 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:24.357809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:24.357847 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:24.358298 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:24.358303 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:24.360340 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
W20260812 14:18:24.360482 19253 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:24.355848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:50685 (local address 127.8.85.62:45087)
0812 14:18:24.356016 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:24.356018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:24.356027 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:18:24.356049 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:24.356050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:24.356059 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:24.356125 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:24.356421 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:24.356908 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:24.357562 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:24.357745 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:24.357795 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:24.360415 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:50685: BlockingRecv error: recv got EOF from 127.8.85.1:50685 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":106,"threads_started":1}
Aug 12 14:18:24 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:24 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:24.488481 19254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:24.483902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:44297)
0812 14:18:24.484061 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:24.484071 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:24.484136 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:24.484259 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:24.484260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:24.484492 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:24.484551 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:24.484553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:24.485195 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:24.485196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:24.485770 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:24.485772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:24.486088 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:24.486089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:24.486126 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:24.486571 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:24.486576 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:24.488408 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":83,"threads_started":1}
W20260812 14:18:24.488597 19253 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:24.483977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:44297 (local address 127.8.85.62:45087)
0812 14:18:24.484081 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:24.484082 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:24.484120 (+ 38us) server_negotiation.cc:407] Connection header received
0812 14:18:24.484189 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:24.484190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:24.484200 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:24.484241 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:24.484622 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:24.485125 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:24.485837 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:24.486024 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:24.486079 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:24.488504 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:44297: BlockingRecv error: recv got EOF from 127.8.85.2:44297 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 14:18:24 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:24 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:24.637439 19255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:24.632853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:35561)
0812 14:18:24.632999 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:24.633009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:24.633067 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:24.633194 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:24.633195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:24.633357 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:24.633407 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:24.633408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:24.634040 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:24.634041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:24.634597 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:24.634599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:24.634983 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:24.634985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:24.635052 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:24.635544 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:24.635549 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:24.637369 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
W20260812 14:18:24.637539 19253 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:24.632943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:35561 (local address 127.8.85.62:45087)
0812 14:18:24.633006 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:24.633008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:24.633051 (+ 43us) server_negotiation.cc:407] Connection header received
0812 14:18:24.633111 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:24.633112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:24.633125 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:24.633175 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:24.633467 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:24.633975 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:24.634672 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:24.634864 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:24.634943 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:24.637469 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:35561: BlockingRecv error: recv got EOF from 127.8.85.3:35561 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 12 14:18:25 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:25 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:25.365376 19256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:25.360967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:54939)
0812 14:18:25.361109 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:25.361120 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:25.361170 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:25.361290 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:25.361290 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:25.361454 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:25.361506 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:25.361507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:25.362157 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:25.362158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:25.362676 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:25.362677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:25.363009 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:25.363010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:25.363050 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:25.363530 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:25.363536 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:25.365317 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
W20260812 14:18:25.365504 19257 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:25.361056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:54939 (local address 127.8.85.62:45087)
0812 14:18:25.361166 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:25.361168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:25.361202 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:18:25.361224 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:25.361224 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:25.361234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:25.361277 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:25.361568 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:25.362087 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:25.362746 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:25.362946 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:25.362997 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:25.365416 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:54939: BlockingRecv error: recv got EOF from 127.8.85.1:54939 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
Aug 12 14:18:25 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:25 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:25.493996 19258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:25.488960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:39879)
0812 14:18:25.489159 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:25.489209 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:25.489324 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:25.489495 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:25.489496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:25.489699 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:25.489748 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:25.489750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:25.490540 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:25.490541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:25.491124 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:25.491126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:25.491627 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:25.491628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:25.491698 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:25.492152 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:25.492157 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:25.493937 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":108,"threads_started":1}
W20260812 14:18:25.494112 19257 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:25.489046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:39879 (local address 127.8.85.62:45087)
0812 14:18:25.489119 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:25.489121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:25.489262 (+ 141us) server_negotiation.cc:407] Connection header received
0812 14:18:25.489409 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:25.489410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:25.489424 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:25.489478 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:25.489896 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:25.490444 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:25.491216 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:25.491514 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:25.491625 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:25.494033 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:39879: BlockingRecv error: recv got EOF from 127.8.85.2:39879 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 14:18:25 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:25 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:25.643108 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:25.638012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:56475)
0812 14:18:25.638159 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:25.638170 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:25.638242 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:25.638400 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:25.638401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:25.638577 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:25.638625 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:25.638627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:25.639328 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:25.639329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:25.639918 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:25.639920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:25.640311 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:25.640312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:25.640380 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:25.640927 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:25.640932 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:25.643050 (+ 2118us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":61,"threads_started":1}
W20260812 14:18:25.643244 19257 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:25.638196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:56475 (local address 127.8.85.62:45087)
0812 14:18:25.638268 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:25.638270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:25.638280 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:18:25.638307 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:25.638308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:25.638322 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:25.638380 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:25.638718 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:25.639239 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:25.640021 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:25.640232 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:25.640312 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:25.643153 (+ 2841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:56475: BlockingRecv error: recv got EOF from 127.8.85.3:56475 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 14:18:26 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:26 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:26.377430 19260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:26.366118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:53965)
0812 14:18:26.366275 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:26.366287 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:26.366370 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:26.366577 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:26.366578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:26.366754 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:26.366805 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:26.366807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:26.367625 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:26.367626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:26.368165 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:26.368166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:26.368547 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:26.368548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:26.368585 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:26.375135 (+ 6550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:26.375141 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:26.377355 (+ 2214us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":79,"threads_started":1}
W20260812 14:18:26.377583 19262 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:26.366194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:53965 (local address 127.8.85.62:45087)
0812 14:18:26.366324 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:26.366327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:26.366338 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:18:26.366462 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:26.366463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:26.366478 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:26.366558 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:26.366898 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:26.367542 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:26.368258 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:26.368476 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:26.368538 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:26.377465 (+ 8927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:53965: BlockingRecv error: recv got EOF from 127.8.85.1:53965 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
Aug 12 14:18:26 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:26 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:26.499236 19263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:26.494585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:46193)
0812 14:18:26.494802 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:26.494815 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:26.494879 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:26.495048 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:26.495049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:26.495246 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:26.495296 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:26.495298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:26.495965 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:26.495966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:26.496475 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:26.496477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:26.496804 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:26.496805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:26.496842 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:26.497289 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:26.497293 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:26.499177 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":91,"threads_started":1}
W20260812 14:18:26.499388 19262 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:26.494649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:46193 (local address 127.8.85.62:45087)
0812 14:18:26.494694 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:26.494697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:26.494889 (+ 192us) server_negotiation.cc:407] Connection header received
0812 14:18:26.494934 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:26.494935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:26.494950 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:26.495036 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:26.495364 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:26.495887 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:26.496550 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:26.496740 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:26.496793 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:26.499300 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:46193: BlockingRecv error: recv got EOF from 127.8.85.2:46193 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 12 14:18:26 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:26 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:26.648811 19264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:26.643702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:55149)
0812 14:18:26.643877 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:26.643888 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:26.643951 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:26.644057 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:26.644058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:26.644230 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:26.644282 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:26.644283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:26.645030 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:26.645031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:26.645604 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:26.645606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:26.645947 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:26.645948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:26.645990 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:26.646549 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:26.646555 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:26.648693 (+ 2138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":73,"threads_started":1}
W20260812 14:18:26.648862 19262 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:26.643784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:55149 (local address 127.8.85.62:45087)
0812 14:18:26.643852 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:26.643859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:26.643961 (+ 102us) server_negotiation.cc:407] Connection header received
0812 14:18:26.643989 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:26.643990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:26.644003 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:26.644051 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:26.644375 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:26.644942 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:26.645692 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:26.645878 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:26.645933 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:26.648794 (+ 2861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:55149: BlockingRecv error: recv got EOF from 127.8.85.3:55149 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 14:18:27 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:27 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:27.382572 19265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:27.377924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:54741)
0812 14:18:27.378070 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:27.378088 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:27.378147 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:27.378295 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:27.378295 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:27.378485 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:27.378535 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:27.378537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:27.379221 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:27.379222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:27.379781 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:27.379783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:27.380128 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:27.380129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:27.380168 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:27.380683 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:27.380688 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:27.382512 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
W20260812 14:18:27.382711 19266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:27.377968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:54741 (local address 127.8.85.62:45087)
0812 14:18:27.378103 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:27.378105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:27.378131 (+ 26us) server_negotiation.cc:407] Connection header received
0812 14:18:27.378204 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:27.378206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:27.378216 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:27.378278 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:27.378610 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:27.379138 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:27.379856 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:27.380061 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:27.380116 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:27.382619 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:54741: BlockingRecv error: recv got EOF from 127.8.85.1:54741 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
Aug 12 14:18:27 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:27 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:27.504860 19268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:27.499731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:50999)
0812 14:18:27.499971 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:27.499983 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:27.500079 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:27.500243 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:27.500245 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:27.500455 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:27.500503 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:27.500505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:27.501198 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:27.501200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:27.501725 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:27.501726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:27.502095 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:27.502096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:27.502156 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:27.502745 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:27.502752 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:27.504746 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":140,"threads_started":1}
W20260812 14:18:27.504901 19266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:27.499858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:50999 (local address 127.8.85.62:45087)
0812 14:18:27.499964 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:27.499966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:27.500058 (+ 92us) server_negotiation.cc:407] Connection header received
0812 14:18:27.500148 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:27.500149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:27.500167 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:27.500237 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:27.500589 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:27.501105 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:27.501820 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:27.502012 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:27.502096 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:27.504839 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:50999: BlockingRecv error: recv got EOF from 127.8.85.2:50999 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 14:18:27 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:27 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:27.653977 19269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:27.649295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:42289)
0812 14:18:27.649508 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:27.649518 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:27.649575 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:27.649704 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:27.649705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:27.649869 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:27.649924 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:27.649926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:27.650575 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:27.650576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:27.651152 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:27.651154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:27.651538 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:27.651539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:27.651581 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:27.652029 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:27.652035 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:27.653874 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":136,"threads_started":1}
W20260812 14:18:27.654011 19266 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:27.649388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:42289 (local address 127.8.85.62:45087)
0812 14:18:27.649449 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:27.649451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:27.649572 (+ 121us) server_negotiation.cc:407] Connection header received
0812 14:18:27.649618 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:27.649619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:27.649630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:27.649680 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:27.649981 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:27.650479 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:27.651227 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:27.651400 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:27.651528 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:27.653951 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:42289: BlockingRecv error: recv got EOF from 127.8.85.3:42289 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 12 14:18:28 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:28 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:28.387743 19270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:28.383061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:47089)
0812 14:18:28.383197 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:28.383214 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:28.383264 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:28.383412 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:28.383413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:28.383592 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:28.383644 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:28.383646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:28.384283 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:28.384284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:28.384908 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:28.384911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:28.385276 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:28.385277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:28.385317 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:28.385829 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:28.385835 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:28.387684 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
W20260812 14:18:28.387851 19271 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:28.383159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:47089 (local address 127.8.85.62:45087)
0812 14:18:28.383280 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:28.383282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:28.383290 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:18:28.383321 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:28.383322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:28.383332 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:28.383392 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:28.383710 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:28.384215 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:28.384996 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:28.385208 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:28.385263 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:28.387775 (+ 2512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:47089: BlockingRecv error: recv got EOF from 127.8.85.1:47089 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
Aug 12 14:18:28 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:28 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:28.510138 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:28.505366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:36611)
0812 14:18:28.505533 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:28.505544 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:28.505626 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:28.505765 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:28.505766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:28.505952 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:28.506000 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:28.506002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:28.506674 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:28.506675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:28.507232 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:28.507234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:28.507677 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:28.507678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:28.507742 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:28.508270 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:28.508274 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:28.510037 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":64,"threads_started":1}
W20260812 14:18:28.510186 19271 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:28.505543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:36611 (local address 127.8.85.62:45087)
0812 14:18:28.505613 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:28.505616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:28.505625 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:18:28.505688 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:28.505689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:28.505702 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:28.505750 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:28.506081 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:28.506589 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:28.507323 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:28.507590 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:28.507664 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:28.510120 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:36611: BlockingRecv error: recv got EOF from 127.8.85.2:36611 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 14:18:28 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:28 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:28.658910 19273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:28.654469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:54145)
0812 14:18:28.654620 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:28.654632 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:28.654698 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:28.654826 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:28.654827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:28.654995 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:28.655046 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:28.655047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:28.655694 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:28.655695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:28.656220 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:28.656222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:28.656527 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:28.656528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:28.656567 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:28.656992 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:28.656997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:28.658817 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
W20260812 14:18:28.658990 19271 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:28.654541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:54145 (local address 127.8.85.62:45087)
0812 14:18:28.654584 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:28.654586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:28.654690 (+ 104us) server_negotiation.cc:407] Connection header received
0812 14:18:28.654741 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:28.654742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:28.654754 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:28.654813 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:28.655106 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:28.655626 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:28.656288 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:28.656464 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:28.656512 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:28.658912 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:54145: BlockingRecv error: recv got EOF from 127.8.85.3:54145 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 12 14:18:29 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:29 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:29.392971 19274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:29.388226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:51859)
0812 14:18:29.388363 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:29.388378 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:29.388432 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:29.388588 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:29.388589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:29.388756 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:29.388843 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:29.388845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:29.389503 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:29.389504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:29.390031 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:29.390032 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:29.390348 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:29.390349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:29.390388 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:29.390819 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:29.390824 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:29.392862 (+ 2038us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":100,"threads_started":1}
W20260812 14:18:29.393011 19275 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:29.388303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:51859 (local address 127.8.85.62:45087)
0812 14:18:29.388431 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:29.388433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:29.388442 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:18:29.388469 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:29.388470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:29.388480 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:29.388542 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:29.388916 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:29.389443 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:29.390101 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:29.390284 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:29.390333 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:29.392943 (+ 2610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:51859: BlockingRecv error: recv got EOF from 127.8.85.1:51859 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
Aug 12 14:18:29 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:29 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:29.515460 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:29.510671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:38897)
0812 14:18:29.510814 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:29.510828 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:29.510924 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:29.511071 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:29.511072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:29.511236 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:29.511286 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:29.511287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:29.512021 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:29.512022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:29.512592 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:29.512594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:29.513027 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:29.513028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:29.513086 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:29.513657 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:29.513661 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:29.515373 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
W20260812 14:18:29.515606 19275 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:29.510758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:38897 (local address 127.8.85.62:45087)
0812 14:18:29.510839 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:29.510841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:29.510902 (+ 61us) server_negotiation.cc:407] Connection header received
0812 14:18:29.510990 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:29.510991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:29.511002 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:29.511051 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:29.511346 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:29.511951 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:29.512665 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:29.512932 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:29.512999 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:29.515538 (+ 2539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:38897: BlockingRecv error: recv got EOF from 127.8.85.2:38897 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 14:18:29 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:29 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:29.664179 19277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:29.659457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:56063)
0812 14:18:29.659619 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:29.659630 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:29.659686 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:29.659814 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:29.659815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:29.660000 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:29.660052 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:29.660054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:29.660681 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:29.660682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:29.661236 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:29.661237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:29.661567 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:29.661569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:29.661619 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:29.662111 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:29.662116 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:29.664117 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
W20260812 14:18:29.664304 19275 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:29.659542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:56063 (local address 127.8.85.62:45087)
0812 14:18:29.659619 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:29.659621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:29.659697 (+ 76us) server_negotiation.cc:407] Connection header received
0812 14:18:29.659724 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:29.659725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:29.659736 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:29.659792 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:29.660117 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:29.660600 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:29.661309 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:29.661481 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:29.661531 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:29.664206 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:56063: BlockingRecv error: recv got EOF from 127.8.85.3:56063 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 14:18:30 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:30 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:30.398555 19279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:30.393489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:46819)
0812 14:18:30.393688 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:30.393699 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:30.393764 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:30.393990 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:30.393991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:30.394195 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:30.394249 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:30.394251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:30.395033 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:30.395034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:30.395718 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:30.395721 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:30.396110 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:30.396111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:30.396161 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:30.396642 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:30.396646 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:30.398495 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":131,"threads_started":1}
W20260812 14:18:30.398680 19280 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:30.393586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:46819 (local address 127.8.85.62:45087)
0812 14:18:30.393701 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:30.393703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:30.393748 (+ 45us) server_negotiation.cc:407] Connection header received
0812 14:18:30.393825 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:30.393826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:30.393838 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:30.393922 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:30.394367 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:30.394944 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:30.395807 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:30.396038 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:30.396097 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:30.398596 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:46819: BlockingRecv error: recv got EOF from 127.8.85.1:46819 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":53,"threads_started":1}
Aug 12 14:18:30 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:30 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:30.521788 19281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:30.515978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:48923)
0812 14:18:30.516177 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:30.516187 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:30.516249 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:30.516407 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:30.516408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:30.516582 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:30.516657 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:30.516660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:30.517596 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:30.517597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:30.518237 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:30.518239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:30.518688 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:30.518689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:30.518757 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:30.519535 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:30.519542 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:30.521726 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":139,"threads_started":1}
W20260812 14:18:30.521938 19280 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:30.516071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:48923 (local address 127.8.85.62:45087)
0812 14:18:30.516181 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:30.516183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:30.516260 (+ 77us) server_negotiation.cc:407] Connection header received
0812 14:18:30.516322 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:30.516323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:30.516336 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:30.516386 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:30.516803 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:30.517513 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:30.518335 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:30.518547 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:30.518697 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:30.521833 (+ 3136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:48923: BlockingRecv error: recv got EOF from 127.8.85.2:48923 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 14:18:30 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:30 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:30.669512 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:30.664728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:56565)
0812 14:18:30.664964 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:30.664976 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:30.665040 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:30.665176 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:30.665177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:30.665346 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:30.665409 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:30.665411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:30.666029 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:30.666030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:30.666574 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:30.666576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:30.666892 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:30.666893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:30.666934 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:30.667403 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:30.667407 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:30.669452 (+ 2045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":127,"threads_started":1}
W20260812 14:18:30.669652 19280 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:30.664798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:56565 (local address 127.8.85.62:45087)
0812 14:18:30.664872 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:30.664874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:30.665041 (+ 167us) server_negotiation.cc:407] Connection header received
0812 14:18:30.665097 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:30.665098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:30.665111 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:30.665153 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:30.665472 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:30.665960 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:30.666648 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:30.666823 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:30.666871 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:30.669567 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:56565: BlockingRecv error: recv got EOF from 127.8.85.3:56565 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 14:18:31 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:31 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:31.410384 19283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:31.399089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:52031)
0812 14:18:31.399361 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:31.399373 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:31.399466 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:31.399612 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:31.399613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:31.399773 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:31.399823 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:31.399825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:31.400471 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:31.400472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:31.400995 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:31.400996 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:31.401346 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:31.401347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:31.401388 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:31.408502 (+ 7114us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:31.408508 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:31.410323 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":208,"threads_started":1}
W20260812 14:18:31.410580 19285 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:31.399326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:52031 (local address 127.8.85.62:45087)
0812 14:18:31.399500 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:31.399502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:31.399512 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:18:31.399535 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:31.399536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:31.399546 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:31.399600 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:31.399894 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:31.400394 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:31.401068 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:31.401274 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:31.401330 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:31.410494 (+ 9164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:52031: BlockingRecv error: recv got EOF from 127.8.85.1:52031 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
Aug 12 14:18:31 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:31 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:31.527155 19286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:31.522312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:44405)
0812 14:18:31.522474 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:31.522485 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:31.522537 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:31.522680 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:31.522680 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:31.522852 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:31.522902 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:31.522903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:31.523800 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:31.523800 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:31.524330 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:31.524331 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:31.524705 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:31.524706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:31.524752 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:31.525214 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:31.525218 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:31.527054 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
W20260812 14:18:31.527207 19285 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:31.522456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:44405 (local address 127.8.85.62:45087)
0812 14:18:31.522517 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:31.522519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:31.522528 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:18:31.522591 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:31.522592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:31.522603 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:31.522663 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:31.522975 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:31.523728 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:31.524434 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:31.524628 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:31.524689 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:31.527138 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:44405: BlockingRecv error: recv got EOF from 127.8.85.2:44405 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 12 14:18:31 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:31 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:31.675060 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:31.670006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:55565)
0812 14:18:31.670194 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:31.670217 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:31.670290 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:31.670446 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:31.670447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:31.670637 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:31.670684 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:31.670685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:31.671354 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:31.671355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:31.672242 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:31.672245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:31.672626 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:31.672627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:31.672683 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:31.673186 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:31.673192 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:31.675000 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
W20260812 14:18:31.675163 19285 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:31.670108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:55565 (local address 127.8.85.62:45087)
0812 14:18:31.670182 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:31.670184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:31.670268 (+ 84us) server_negotiation.cc:407] Connection header received
0812 14:18:31.670346 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:31.670347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:31.670363 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:31.670438 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:31.670768 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:31.671260 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:31.672328 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:31.672522 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:31.672627 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:31.675094 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:55565: BlockingRecv error: recv got EOF from 127.8.85.3:55565 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 14:18:32 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:32 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:32.415966 19289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:32.410930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:42401)
0812 14:18:32.411107 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:32.411118 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:32.411172 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:32.411338 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:32.411339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:32.411535 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:32.411588 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:32.411590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:32.412351 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:32.412352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:32.412924 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:32.412925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:32.413326 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:32.413328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:32.413376 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:32.413868 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:32.413873 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:32.415856 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":105,"threads_started":1}
W20260812 14:18:32.416040 19291 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:32.410998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:42401 (local address 127.8.85.62:45087)
0812 14:18:32.411133 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:32.411140 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:32.411153 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:18:32.411232 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:32.411233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:32.411244 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:32.411305 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:32.411651 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:32.412258 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:32.413028 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:32.413228 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:32.413297 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:32.415936 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:42401: BlockingRecv error: recv got EOF from 127.8.85.1:42401 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
Aug 12 14:18:32 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:32 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:32.532550 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:32.527709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:41549)
0812 14:18:32.527860 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:32.527871 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:32.527934 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:32.528094 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:32.528095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:32.528275 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:32.528332 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:32.528334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:32.529046 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:32.529047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:32.529580 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:32.529582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:32.529953 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:32.529954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:32.530003 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:32.530508 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:32.530513 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:32.532482 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":135,"threads_started":1}
W20260812 14:18:32.532657 19291 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:32.527774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:41549 (local address 127.8.85.62:45087)
0812 14:18:32.527844 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:32.527847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:32.527936 (+ 89us) server_negotiation.cc:407] Connection header received
0812 14:18:32.527986 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:32.527987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:32.527998 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:32.528051 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:32.528457 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:32.528961 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:32.529666 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:32.529862 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:32.529930 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:32.532578 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:41549: BlockingRecv error: recv got EOF from 127.8.85.2:41549 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 14:18:32 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:32 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:32.680428 19293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:32.675592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:60901)
0812 14:18:32.675751 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:32.675762 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:32.675821 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:32.675931 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:32.675931 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:32.676101 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:32.676148 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:32.676149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:32.676869 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:32.676870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:32.677494 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:32.677497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:32.677963 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:32.677965 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:32.678012 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:32.678438 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:32.678442 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:32.680362 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
W20260812 14:18:32.680564 19291 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:32.675675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:60901 (local address 127.8.85.62:45087)
0812 14:18:32.675773 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:32.675776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:32.675813 (+ 37us) server_negotiation.cc:407] Connection header received
0812 14:18:32.675864 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:32.675865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:32.675876 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:32.675922 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:32.676207 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:32.676768 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:32.677589 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:32.677868 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:32.677966 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:32.680464 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:60901: BlockingRecv error: recv got EOF from 127.8.85.3:60901 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 14:18:33 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:33 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:33.421134 19294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:33.416579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:34343)
0812 14:18:33.416733 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:33.416753 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:33.416815 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:33.416938 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:33.416939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:33.417108 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:33.417159 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:33.417160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:33.417812 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:33.417813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:33.418367 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:33.418368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:33.418695 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:33.418696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:33.418734 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:33.419202 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:33.419206 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:33.421076 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
W20260812 14:18:33.421250 19295 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:33.416613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:34343 (local address 127.8.85.62:45087)
0812 14:18:33.416768 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:33.416770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:33.416797 (+ 27us) server_negotiation.cc:407] Connection header received
0812 14:18:33.416860 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:33.416860 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:33.416872 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:33.416931 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:33.417214 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:33.417748 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:33.418447 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:33.418635 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:33.418684 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:33.421170 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:34343: BlockingRecv error: recv got EOF from 127.8.85.1:34343 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
Aug 12 14:18:33 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:33 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:33.537914 19296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:33.533180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:60681)
0812 14:18:33.533320 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:33.533332 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:33.533403 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:33.533601 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:33.533602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:33.533829 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:33.533893 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:33.533895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:33.534541 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:33.534542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:33.535088 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:33.535089 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:33.535487 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:33.535487 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:33.535546 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:33.536052 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:33.536056 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:33.537856 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
W20260812 14:18:33.538022 19295 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:33.533219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:60681 (local address 127.8.85.62:45087)
0812 14:18:33.533288 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:33.533291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:33.533407 (+ 116us) server_negotiation.cc:407] Connection header received
0812 14:18:33.533479 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:33.533479 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:33.533512 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:33.533577 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:33.533965 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:33.534476 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:33.535161 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:33.535366 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:33.535416 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:33.537952 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:60681: BlockingRecv error: recv got EOF from 127.8.85.2:60681 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 14:18:33 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:33 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:33.686666 19297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:33.680881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:54217)
0812 14:18:33.681031 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:33.681043 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:33.681114 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:33.681277 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:33.681278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:33.681492 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:33.681544 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:33.681545 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:33.682225 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:33.682226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:33.682768 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:33.682770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:33.683149 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:33.683150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:33.683209 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:33.683786 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:33.683792 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:33.686555 (+ 2763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":63,"threads_started":1}
W20260812 14:18:33.686715 19295 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:33.681045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:54217 (local address 127.8.85.62:45087)
0812 14:18:33.681104 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:33.681106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:33.681115 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:18:33.681180 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:33.681181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:33.681195 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:33.681271 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:33.681623 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:33.682137 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:33.682863 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:33.683058 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:33.683137 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:33.686647 (+ 3510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:54217: BlockingRecv error: recv got EOF from 127.8.85.3:54217 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 12 14:18:34 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:34 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:34.426462 19299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:34.421742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:40095)
0812 14:18:34.421898 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:34.421908 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:34.421963 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:34.422110 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:34.422110 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:34.422284 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:34.422334 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:34.422336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:34.423008 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:34.423009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:34.423631 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:34.423632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:34.423953 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:34.423954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:34.423995 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:34.424436 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:34.424441 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:34.426378 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
W20260812 14:18:34.426589 19300 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:34.421810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:40095 (local address 127.8.85.62:45087)
0812 14:18:34.421954 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:34.421956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:34.421966 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:18:34.422024 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:34.422025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:34.422035 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:34.422087 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:34.422411 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:34.422910 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:34.423706 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:34.423890 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:34.423950 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:34.426483 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:40095: BlockingRecv error: recv got EOF from 127.8.85.1:40095 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
Aug 12 14:18:34 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:34 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:34.543310 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:34.538425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:59805)
0812 14:18:34.538571 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:34.538583 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:34.538649 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:34.538804 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:34.538805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:34.538992 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:34.539043 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:34.539045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:34.539855 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:34.539856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:34.540381 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:34.540383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:34.540741 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:34.540742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:34.540781 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:34.541296 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:34.541300 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:34.543233 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
W20260812 14:18:34.543462 19300 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:34.538523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:59805 (local address 127.8.85.62:45087)
0812 14:18:34.538596 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:34.538603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:34.538637 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:18:34.538707 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:34.538708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:34.538719 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:34.538789 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:34.539147 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:34.539773 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:34.540462 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:34.540670 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:34.540731 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:34.543353 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:59805: BlockingRecv error: recv got EOF from 127.8.85.2:59805 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 14:18:34 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:34 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:34.692274 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:34.687180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:58411)
0812 14:18:34.687381 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:34.687394 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:34.687507 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:34.687695 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:34.687696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:34.687894 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:34.687939 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:34.687941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:34.688662 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:34.688663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:34.689314 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:34.689318 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:34.689818 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:34.689819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:34.689886 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:34.690402 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:34.690406 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:34.692217 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":106,"threads_started":1}
W20260812 14:18:34.692408 19300 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:34.687283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:58411 (local address 127.8.85.62:45087)
0812 14:18:34.687366 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:34.687368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:34.687486 (+ 118us) server_negotiation.cc:407] Connection header received
0812 14:18:34.687593 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:34.687595 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:34.687615 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:34.687676 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:34.688018 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:34.688570 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:34.689434 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:34.689696 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:34.689795 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:34.692311 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:58411: BlockingRecv error: recv got EOF from 127.8.85.3:58411 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 14:18:35 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:35 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:35.431550 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:35.426931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:45429)
0812 14:18:35.427122 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:35.427132 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:35.427181 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:35.427321 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:35.427322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:35.427522 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:35.427573 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:35.427575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:35.428211 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:35.428212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:35.428732 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:35.428733 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:35.429060 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:35.429061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:35.429100 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:35.429643 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:35.429648 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:35.431412 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":107,"threads_started":1}
W20260812 14:18:35.431610 19304 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:35.427025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:45429 (local address 127.8.85.62:45087)
0812 14:18:35.427163 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:35.427165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:35.427173 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:18:35.427227 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:35.427228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:35.427239 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:35.427302 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:35.427639 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:35.428144 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:35.428805 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:35.428994 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:35.429044 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:35.431532 (+ 2488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:45429: BlockingRecv error: recv got EOF from 127.8.85.1:45429 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
Aug 12 14:18:35 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:35 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:35.548774 19305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:35.543952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:37945)
0812 14:18:35.544175 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:35.544187 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:35.544260 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:35.544412 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:35.544413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:35.544596 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:35.544649 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:35.544651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:35.545290 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:35.545291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:35.545876 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:35.545878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:35.546210 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:35.546211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:35.546253 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:35.546758 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:35.546763 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:35.548700 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":74,"threads_started":1}
W20260812 14:18:35.549070 19304 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:35.544030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:37945 (local address 127.8.85.62:45087)
0812 14:18:35.544087 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:35.544090 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:35.544246 (+ 156us) server_negotiation.cc:407] Connection header received
0812 14:18:35.544308 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:35.544309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:35.544320 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:35.544369 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:35.544716 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:35.545216 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:35.545958 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:35.546141 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:35.546192 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:35.548787 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:37945: BlockingRecv error: recv got EOF from 127.8.85.2:37945 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 12 14:18:35 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:35 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:35.697497 19306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:35.692775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:46631)
0812 14:18:35.692934 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:35.692947 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:35.693012 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:35.693165 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:35.693166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:35.693345 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:35.693420 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:35.693422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:35.694071 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:35.694072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:35.694682 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:35.694684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:35.695165 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:35.695166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:35.695230 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:35.695701 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:35.695706 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:35.697431 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
W20260812 14:18:35.697641 19304 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:35.692916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:46631 (local address 127.8.85.62:45087)
0812 14:18:35.692988 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:35.692990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:35.692999 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:18:35.693068 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:35.693069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:35.693082 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:35.693127 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:35.693483 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:35.694006 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:35.694764 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:35.695057 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:35.695141 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:35.697519 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:46631: BlockingRecv error: recv got EOF from 127.8.85.3:46631 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 14:18:36 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:36 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:36.442529 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:36.432047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:49111)
0812 14:18:36.432278 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:36.432289 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:36.432384 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:36.432540 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:36.432540 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:36.432730 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:36.432784 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:36.432786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:36.433602 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:36.433603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:36.434491 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:36.434494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:36.435007 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:36.435008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:36.435067 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:36.440123 (+ 5056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:36.440130 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:36.442430 (+ 2300us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":77,"threads_started":1}
W20260812 14:18:36.442605 19308 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:36.432106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:49111 (local address 127.8.85.62:45087)
0812 14:18:36.432281 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:36.432285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:36.432362 (+ 77us) server_negotiation.cc:407] Connection header received
0812 14:18:36.432453 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:36.432455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:36.432470 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:36.432532 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:36.432879 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:36.433531 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:36.434578 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:36.434924 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:36.435006 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:36.442547 (+ 7541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:49111: BlockingRecv error: recv got EOF from 127.8.85.1:49111 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
Aug 12 14:18:36 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:36 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:36.554289 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:36.549267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:55367)
0812 14:18:36.549413 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:36.549423 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:36.549515 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:36.549693 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:36.549693 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:36.549891 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:36.549940 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:36.549942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:36.550630 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:36.550631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:36.551197 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:36.551199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:36.551643 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:36.551645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:36.551710 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:36.552237 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:36.552242 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:36.554209 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
W20260812 14:18:36.554399 19308 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:36.549406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:55367 (local address 127.8.85.62:45087)
0812 14:18:36.549481 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:36.549483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:36.549495 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:18:36.549599 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:36.549600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:36.549620 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:36.549682 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:36.550024 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:36.550538 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:36.551289 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:36.551553 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:36.551641 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:36.554308 (+ 2667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:55367: BlockingRecv error: recv got EOF from 127.8.85.2:55367 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 14:18:36 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:36 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:36.702862 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:36.697986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:57057)
0812 14:18:36.698154 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:36.698167 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:36.698261 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:36.698416 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:36.698417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:36.698611 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:36.698659 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:36.698661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:36.699372 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:36.699373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:36.699967 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:36.699968 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:36.700354 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:36.700355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:36.700420 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:36.700915 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:36.700924 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:36.702753 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
W20260812 14:18:36.702893 19308 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:36.698076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:57057 (local address 127.8.85.62:45087)
0812 14:18:36.698151 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:36.698153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:36.698239 (+ 86us) server_negotiation.cc:407] Connection header received
0812 14:18:36.698337 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:36.698338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:36.698353 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:36.698401 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:36.698748 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:36.699263 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:36.700064 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:36.700268 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:36.700354 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:36.702826 (+ 2472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:57057: BlockingRecv error: recv got EOF from 127.8.85.3:57057 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 14:18:37 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:37 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:37.447638 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:37.443088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:54071)
0812 14:18:37.443241 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:37.443251 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:37.443306 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:37.443447 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:37.443448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:37.443612 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:37.443662 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:37.443664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:37.444319 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:37.444320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:37.444876 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:37.444878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:37.445214 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:37.445215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:37.445254 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:37.445716 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:37.445720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:37.447566 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
W20260812 14:18:37.447764 19312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:37.443173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:54071 (local address 127.8.85.62:45087)
0812 14:18:37.443298 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:37.443299 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:37.443309 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:18:37.443342 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:37.443342 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:37.443352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:37.443409 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:37.443730 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:37.444249 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:37.444958 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:37.445148 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:37.445200 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:37.447677 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:54071: BlockingRecv error: recv got EOF from 127.8.85.1:54071 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":55,"threads_started":1}
Aug 12 14:18:37 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:37 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:37.559844 19313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:37.554904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:56357)
0812 14:18:37.555108 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:37.555120 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:37.555199 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:37.555374 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:37.555375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:37.555594 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:37.555641 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:37.555643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:37.556344 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:37.556345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:37.556915 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:37.556917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:37.557371 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:37.557372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:37.557439 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:37.557944 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:37.557949 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:37.559771 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":126,"threads_started":1}
W20260812 14:18:37.559947 19312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:37.555011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:56357 (local address 127.8.85.62:45087)
0812 14:18:37.555082 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:37.555084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:37.555189 (+ 105us) server_negotiation.cc:407] Connection header received
0812 14:18:37.555278 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:37.555279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:37.555297 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:37.555364 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:37.555738 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:37.556250 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:37.557030 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:37.557290 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:37.557363 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:37.559871 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:56357: BlockingRecv error: recv got EOF from 127.8.85.2:56357 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 12 14:18:37 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:37 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.3@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:37.708395 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:37.703508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.3:37989)
0812 14:18:37.703707 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:37.703719 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:37.703807 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:37.703985 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:37.703985 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:37.704169 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:37.704215 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:37.704217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:37.704911 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:37.704912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:37.705462 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:37.705464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:37.705823 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:37.705824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:37.705884 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:37.706405 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:37.706410 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:37.708281 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":101,"threads_started":1}
W20260812 14:18:37.708427 19312 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:37.703599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:37989 (local address 127.8.85.62:45087)
0812 14:18:37.703687 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:37.703689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:37.703791 (+ 102us) server_negotiation.cc:407] Connection header received
0812 14:18:37.703867 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:37.703868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:37.703884 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:37.703970 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:37.704300 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:37.704815 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:37.705554 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:37.705747 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:37.705825 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:37.708360 (+ 2535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.3:37989: BlockingRecv error: recv got EOF from 127.8.85.3:37989 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 14:18:38 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:38 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.1@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:38.453295 19316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:38.448285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.1:52487)
0812 14:18:38.448415 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:38.448429 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:38.448498 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:38.448674 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:38.448675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:38.448846 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:38.448899 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:38.448900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:38.449701 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:38.449702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:38.450232 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:38.450234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:38.450577 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:38.450578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:38.450618 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:38.451059 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:38.451064 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:38.453197 (+ 2133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
W20260812 14:18:38.453343 19317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:38.448353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:52487 (local address 127.8.85.62:45087)
0812 14:18:38.448478 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:38.448480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:38.448491 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:18:38.448580 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:38.448581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:38.448593 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:38.448666 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:38.449028 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:38.449609 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:38.450312 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:38.450509 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:38.450563 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:38.453275 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.1:52487: BlockingRecv error: recv got EOF from 127.8.85.1:52487 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
Aug 12 14:18:38 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:18:38 dist-test-slave-5k54 krb5kdc[18096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.85.2@KRBTEST.COM for oryx/127.8.85.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:18:38.565683 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:38.560492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45087 (local address 127.8.85.2:59217)
0812 14:18:38.560651 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:38.560663 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:38.560736 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:38.560948 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:38.560949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:38.561189 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:38.561274 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:38.561277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:38.562120 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:38.562121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:38.562738 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:38.562740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:38.563163 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:38.563164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:38.563230 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:38.563710 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:38.563714 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:38.565587 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.85.62:45087: Server oryx/127.8.85.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
W20260812 14:18:38.565745 19317 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:38.560565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:59217 (local address 127.8.85.62:45087)
0812 14:18:38.560652 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:38.560654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:38.560719 (+ 65us) server_negotiation.cc:407] Connection header received
0812 14:18:38.560817 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:38.560818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:38.560832 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:38.560923 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:38.561379 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:38.562010 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:38.562842 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:38.563066 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:38.563150 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:38.565680 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.85.2:59217: BlockingRecv error: recv got EOF from 127.8.85.2:59217 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 14:18:38.625440 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 18768
I20260812 14:18:38.633239 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 18943
I20260812 14:18:38.638785 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 19100
I20260812 14:18:38.643595 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 18691
2026-08-12T14:18:38Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17577 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T14:18:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:38Z Disabled control of system clock
I20260812 14:18:38.674752 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:44465
--webserver_interface=127.8.85.62
--webserver_port=0
--builtin_ntp_servers=127.8.85.20:44995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:44465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 14:18:38.758988 19322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:38.759192 19322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:38.759212 19322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:38.760409 19322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:38.760452 19322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:38.760465 19322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:38.760478 19322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:38.760490 19322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:38.761770 19322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:44995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:44465
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:44465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:38.761966 19322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:38.762205 19322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:38.766981 19335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:38.767016 19333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:38.766992 19331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:38.767262 19322 server_base.cc:1034] running on GCE node
I20260812 14:18:38.767572 19322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:38.767867 19322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:38.769014 19322 hybrid_clock.cc:648] HybridClock initialized: now 1786544318768982 us; error 47 us; skew 500 ppm
I20260812 14:18:38.770197 19322 webserver.cc:460] Webserver started at http://127.8.85.62:37087/ using document root <none> and password file <none>
I20260812 14:18:38.770463 19322 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:38.770510 19322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:38.770686 19322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:38.771857 19322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "7e7ad63396da499ea5860d2038673fa8"
format_stamp: "Formatted at 2026-08-12 14:18:38 on dist-test-slave-5k54"
server_key: "9dfa8fdf6da545bb7ff2ffbaef409eff"
server_key_iv: "995e35f581a0a2a29a3723dc3981c144"
server_key_version: "encryptionkey@0"
I20260812 14:18:38.772243 19322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "7e7ad63396da499ea5860d2038673fa8"
format_stamp: "Formatted at 2026-08-12 14:18:38 on dist-test-slave-5k54"
server_key: "9dfa8fdf6da545bb7ff2ffbaef409eff"
server_key_iv: "995e35f581a0a2a29a3723dc3981c144"
server_key_version: "encryptionkey@0"
I20260812 14:18:38.773607 19322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:38.774338 19341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:38.774520 19322 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:38.774624 19322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "7e7ad63396da499ea5860d2038673fa8"
format_stamp: "Formatted at 2026-08-12 14:18:38 on dist-test-slave-5k54"
server_key: "9dfa8fdf6da545bb7ff2ffbaef409eff"
server_key_iv: "995e35f581a0a2a29a3723dc3981c144"
server_key_version: "encryptionkey@0"
I20260812 14:18:38.774708 19322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:38.800010 19322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:38.800381 19322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:38.800557 19322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:38.804980 19322 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:44465
I20260812 14:18:38.805020 19399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:44465 every 8 connection(s)
I20260812 14:18:38.805420 19322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:38.806134 19400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:38.808916 19400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8: Bootstrap starting.
I20260812 14:18:38.809365 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 19322
I20260812 14:18:38.809482 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:38.809617 19400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:38.809726 8532 external_mini_cluster.cc:1428] Setting key b7d0a5f5478f6f9155d8d590c56ab4d5
I20260812 14:18:38.809998 19400 log.cc:824] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:38.810770 19400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8: No bootstrap required, opened a new log
I20260812 14:18:38.812340 19400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7ad63396da499ea5860d2038673fa8" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44465 } }
I20260812 14:18:38.812502 19400 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:38.812573 19400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e7ad63396da499ea5860d2038673fa8, State: Initialized, Role: FOLLOWER
I20260812 14:18:38.812736 19400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7ad63396da499ea5860d2038673fa8" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44465 } }
I20260812 14:18:38.813190 19400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:38.813246 19400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:38.813295 19400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:38.813971 19400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7ad63396da499ea5860d2038673fa8" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44465 } }
I20260812 14:18:38.814064 19400 leader_election.cc:302] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e7ad63396da499ea5860d2038673fa8; no voters:
I20260812 14:18:38.814306 19400 leader_election.cc:288] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:38.814354 19405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:38.814563 19405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [term 1 LEADER]: Becoming Leader. State: Replica: 7e7ad63396da499ea5860d2038673fa8, State: Running, Role: LEADER
I20260812 14:18:38.814670 19400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:38.815080 19405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7ad63396da499ea5860d2038673fa8" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44465 } }
I20260812 14:18:38.815511 19406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e7ad63396da499ea5860d2038673fa8. Latest consensus state: current_term: 1 leader_uuid: "7e7ad63396da499ea5860d2038673fa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7ad63396da499ea5860d2038673fa8" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44465 } } }
I20260812 14:18:38.815608 19406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:38.815851 19407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e7ad63396da499ea5860d2038673fa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e7ad63396da499ea5860d2038673fa8" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44465 } } }
I20260812 14:18:38.815966 19407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:38.816138 19404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:38.810277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49246 (local address 127.8.85.62:44465)
0812 14:18:38.810615 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:38.810623 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:38.810658 (+ 35us) server_negotiation.cc:407] Connection header received
0812 14:18:38.811136 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:38.811144 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:38.811206 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:38.811367 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:38.811656 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:38.812569 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:38.813716 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:38.814228 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:38.814360 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:38.815158 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:38.815163 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:38.815163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:38.815359 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:38.815422 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:38.815456 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:38.815520 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:38.815799 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 14:18:38.815882 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":89,"threads_started":1}
I20260812 14:18:38.816156 19415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:38.816666 19415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:38.818220 19415 catalog_manager.cc:1347] Generated new cluster ID: 3edae90f71eb4ab3b22c988cbebd5c65
I20260812 14:18:38.818274 19415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:18:38.837644 19415 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:38.838270 19415 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:38.849102 19415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e7ad63396da499ea5860d2038673fa8: Generated new TSK 0
I20260812 14:18:38.849323 19415 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:18:38.860705 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:44465
--builtin_ntp_servers=127.8.85.20:44995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 14:18:38.947551 19426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:38.947744 19426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:38.947767 19426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:38.948946 19426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:38.948988 19426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:38.949013 19426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:18:38.950328 19426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:44995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:44465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:38.950575 19426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:38.950862 19426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:38.955746 19434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:38.955729 19437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:38.955714 19432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:38.956033 19426 server_base.cc:1034] running on GCE node
I20260812 14:18:38.956257 19426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:38.956497 19426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:38.957659 19426 hybrid_clock.cc:648] HybridClock initialized: now 1786544318957640 us; error 51 us; skew 500 ppm
I20260812 14:18:38.958837 19426 webserver.cc:460] Webserver started at http://127.8.85.1:40255/ using document root <none> and password file <none>
I20260812 14:18:38.959069 19426 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:38.959136 19426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:38.959271 19426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:38.960419 19426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "b608e21d380741f3973a602aa360fac9"
format_stamp: "Formatted at 2026-08-12 14:18:38 on dist-test-slave-5k54"
server_key: "8ebb4bdea76011b778768a0ca6087a92"
server_key_iv: "889c8b1d27046e7ead3fc8ad826ff474"
server_key_version: "encryptionkey@0"
I20260812 14:18:38.960803 19426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "b608e21d380741f3973a602aa360fac9"
format_stamp: "Formatted at 2026-08-12 14:18:38 on dist-test-slave-5k54"
server_key: "8ebb4bdea76011b778768a0ca6087a92"
server_key_iv: "889c8b1d27046e7ead3fc8ad826ff474"
server_key_version: "encryptionkey@0"
I20260812 14:18:38.962143 19426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:38.962815 19444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:38.962977 19426 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:38.963075 19426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "b608e21d380741f3973a602aa360fac9"
format_stamp: "Formatted at 2026-08-12 14:18:38 on dist-test-slave-5k54"
server_key: "8ebb4bdea76011b778768a0ca6087a92"
server_key_iv: "889c8b1d27046e7ead3fc8ad826ff474"
server_key_version: "encryptionkey@0"
I20260812 14:18:38.963158 19426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:38.972360 19426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:38.972628 19426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:38.972769 19426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:38.972985 19426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:38.973290 19426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:38.973343 19426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:38.973382 19426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:38.973410 19426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:38.979424 19426 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:35245
I20260812 14:18:38.979487 19574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:35245 every 8 connection(s)
I20260812 14:18:38.979799 19426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:18:38.984478 19404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:38.980403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:36141 (local address 127.8.85.62:44465)
0812 14:18:38.980484 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:38.980486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:38.980709 (+ 223us) server_negotiation.cc:407] Connection header received
0812 14:18:38.981152 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:38.981153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:38.981162 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:38.981218 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:38.981820 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:38.982311 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:38.983218 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:38.983390 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:38.983492 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:38.984281 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:38.984282 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:38.984282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:38.984286 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:38.984295 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:38.984298 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:38.984335 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:38.984397 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 14:18:38.984415 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 14:18:38.984695 19578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:38.980405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44465 (local address 127.8.85.1:36141)
0812 14:18:38.980586 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:38.980624 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:38.981070 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:38.981266 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:38.981269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:38.981277 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:38.981752 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:38.981756 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:38.982405 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:38.982406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:38.983146 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:38.983148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:38.983505 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:38.983506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:38.983608 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:38.984159 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:18:38.984160 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:38.984186 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:18:38.984208 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:38.984336 (+ 128us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:38.984339 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:38.984341 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:18:38.984385 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:18:38.984463 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260812 14:18:38.985255 19576 heartbeater.cc:347] Connected to a master server at 127.8.85.62:44465
I20260812 14:18:38.985363 19576 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:38.985546 19576 heartbeater.cc:511] Master 127.8.85.62:44465 requested a full tablet report, sending...
I20260812 14:18:38.986033 19363 ts_manager.cc:194] Registered new tserver with Master: b608e21d380741f3973a602aa360fac9 (127.8.85.1:35245)
I20260812 14:18:38.986610 19363 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.85.1:36141
I20260812 14:18:38.986646 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 19426
I20260812 14:18:38.986778 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:18:38.986974 8532 external_mini_cluster.cc:1428] Setting key a49161f48d4a3b9d525ca0268c2250b8
I20260812 14:18:38.988029 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:44465
--builtin_ntp_servers=127.8.85.20:44995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 14:18:39.070587 19579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:39.070791 19579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:39.070812 19579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:39.071990 19579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:39.072031 19579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:39.072080 19579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:18:39.073387 19579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:44995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:44465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:39.073638 19579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:39.073957 19579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:39.078970 19584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:39.079007 19585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:39.078995 19587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:39.079252 19579 server_base.cc:1034] running on GCE node
I20260812 14:18:39.079527 19579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:39.079816 19579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:39.080986 19579 hybrid_clock.cc:648] HybridClock initialized: now 1786544319080993 us; error 51 us; skew 500 ppm
I20260812 14:18:39.082182 19579 webserver.cc:460] Webserver started at http://127.8.85.2:44059/ using document root <none> and password file <none>
I20260812 14:18:39.082450 19579 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:39.082521 19579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:39.082686 19579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:39.083877 19579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "7308b823004c47d9a6de5952a8c83de8"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "9fcbd7092d99eef8870bb1f84730ab03"
server_key_iv: "647961578edf182bd4584b4cf5d2e12d"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.084251 19579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "7308b823004c47d9a6de5952a8c83de8"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "9fcbd7092d99eef8870bb1f84730ab03"
server_key_iv: "647961578edf182bd4584b4cf5d2e12d"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.085714 19579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:39.086437 19593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.086630 19579 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:39.086732 19579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "7308b823004c47d9a6de5952a8c83de8"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "9fcbd7092d99eef8870bb1f84730ab03"
server_key_iv: "647961578edf182bd4584b4cf5d2e12d"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.086815 19579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:39.102660 19579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:39.102963 19579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:39.103123 19579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:39.103355 19579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:39.103736 19579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:39.103793 19579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.103837 19579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:39.103864 19579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.109483 19579 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:43931
I20260812 14:18:39.109560 19718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:43931 every 8 connection(s)
I20260812 14:18:39.109899 19579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:18:39.113400 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 19579
I20260812 14:18:39.113493 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:18:39.113685 8532 external_mini_cluster.cc:1428] Setting key b5e1fd2307b3c4d2ad219bd26d1a8129
I20260812 14:18:39.114830 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:44465
--builtin_ntp_servers=127.8.85.20:44995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 14:18:39.115473 19404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:39.110654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:42399 (local address 127.8.85.62:44465)
0812 14:18:39.110724 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:39.110727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:39.110973 (+ 246us) server_negotiation.cc:407] Connection header received
0812 14:18:39.111483 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:39.111484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:39.111496 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:39.111560 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:39.112180 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:39.112670 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:39.113910 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:39.114164 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:39.114217 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:39.115158 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:39.115160 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:39.115160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:39.115166 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:39.115184 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:39.115189 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:39.115246 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:39.115343 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 14:18:39.115367 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:18:39.115821 19722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:39.110638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44465 (local address 127.8.85.2:42399)
0812 14:18:39.110849 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:39.110882 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:39.111350 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:39.111615 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:39.111619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:39.111628 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:39.112103 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:39.112107 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:39.112748 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:39.112748 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:39.113813 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:39.113816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:39.114240 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:39.114241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:39.114385 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:39.114994 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:18:39.114994 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:39.115033 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:18:39.115063 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:39.115265 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:39.115270 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:39.115273 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:18:39.115348 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 14:18:39.115499 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":108,"threads_started":1}
I20260812 14:18:39.116382 19719 heartbeater.cc:347] Connected to a master server at 127.8.85.62:44465
I20260812 14:18:39.116489 19719 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:39.116699 19719 heartbeater.cc:511] Master 127.8.85.62:44465 requested a full tablet report, sending...
I20260812 14:18:39.117100 19363 ts_manager.cc:194] Registered new tserver with Master: 7308b823004c47d9a6de5952a8c83de8 (127.8.85.2:43931)
I20260812 14:18:39.117482 19363 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.85.2:42399
W20260812 14:18:39.198978 19724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:39.199174 19724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:39.199195 19724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:39.200393 19724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:39.200435 19724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:39.200460 19724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:18:39.201764 19724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:44995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:44465
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:39.202004 19724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:39.202294 19724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:39.207271 19730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:39.207279 19731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:39.207319 19734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:39.207530 19724 server_base.cc:1034] running on GCE node
I20260812 14:18:39.207773 19724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:39.207978 19724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:39.209146 19724 hybrid_clock.cc:648] HybridClock initialized: now 1786544319209126 us; error 30 us; skew 500 ppm
I20260812 14:18:39.210538 19724 webserver.cc:460] Webserver started at http://127.8.85.3:38117/ using document root <none> and password file <none>
I20260812 14:18:39.210772 19724 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:39.210840 19724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:39.211019 19724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:39.212172 19724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "a8860c856e354270a825eef46dc4fe2f"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "feb53e0a67ac88939ea9280008fa5f5c"
server_key_iv: "9e46dc3581d60d53ba8947b0124a721e"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.212550 19724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "a8860c856e354270a825eef46dc4fe2f"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "feb53e0a67ac88939ea9280008fa5f5c"
server_key_iv: "9e46dc3581d60d53ba8947b0124a721e"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.213882 19724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:39.214636 19740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.214810 19724 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:39.214912 19724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "a8860c856e354270a825eef46dc4fe2f"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "feb53e0a67ac88939ea9280008fa5f5c"
server_key_iv: "9e46dc3581d60d53ba8947b0124a721e"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.215008 19724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:39.244756 19724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:39.245131 19724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:39.245296 19724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:39.245555 19724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:39.245926 19724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:39.245993 19724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.246034 19724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:39.246080 19724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.252127 19724 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:35635
I20260812 14:18:39.252175 19876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:35635 every 8 connection(s)
I20260812 14:18:39.252518 19724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:18:39.257885 19404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:39.253236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:41815 (local address 127.8.85.62:44465)
0812 14:18:39.253283 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:39.253286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:39.253547 (+ 261us) server_negotiation.cc:407] Connection header received
0812 14:18:39.254123 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:39.254126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:39.254140 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:39.254192 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:39.254917 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:39.255468 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:39.256471 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:39.256674 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:39.256752 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:39.257592 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:39.257594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:39.257595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:39.257600 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:39.257615 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:39.257620 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:39.257674 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:39.257775 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 14:18:39.257804 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:18:39.258109 19879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:39.253218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44465 (local address 127.8.85.3:41815)
0812 14:18:39.253414 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:39.253455 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:39.253998 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:39.254267 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:39.254271 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:39.254289 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:39.254805 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:39.254810 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:39.255549 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:39.255550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:39.256352 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:39.256354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:39.256760 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:39.256762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:39.256901 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:39.257433 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:18:39.257433 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:39.257478 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:18:39.257503 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:39.257679 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:39.257683 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:39.257687 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:39.257766 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 14:18:39.257875 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260812 14:18:39.258574 19877 heartbeater.cc:347] Connected to a master server at 127.8.85.62:44465
I20260812 14:18:39.258665 19877 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:39.258816 19877 heartbeater.cc:511] Master 127.8.85.62:44465 requested a full tablet report, sending...
I20260812 14:18:39.259208 19363 ts_manager.cc:194] Registered new tserver with Master: a8860c856e354270a825eef46dc4fe2f (127.8.85.3:35635)
I20260812 14:18:39.259617 19363 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.85.3:41815
I20260812 14:18:39.261222 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 19724
I20260812 14:18:39.261296 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:18:39.261452 8532 external_mini_cluster.cc:1428] Setting key d49f14204d86a2b9b483022a22d07576
I20260812 14:18:39.262312 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 14:18:39.264058 19404 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:39.263612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49260 (local address 127.8.85.62:44465)
0812 14:18:39.263655 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:39.263657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:39.263728 (+ 71us) server_negotiation.cc:407] Connection header received
0812 14:18:39.263808 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:39.263809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:39.263819 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:39.263867 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:39.263988 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49260: BlockingRecv error: recv got EOF from 127.0.0.1:49260 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:18:39.264145 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 19426
I20260812 14:18:39.273675 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 19579
I20260812 14:18:39.281298 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 19724
I20260812 14:18:39.290012 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 19322
2026-08-12T14:18:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (645 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T14:18:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:39Z Disabled control of system clock
I20260812 14:18:39.319923 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:46565
--webserver_interface=127.8.85.62
--webserver_port=0
--builtin_ntp_servers=127.8.85.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:46565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 14:18:39.402037 19891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:39.402236 19891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:39.402258 19891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:39.403306 19891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:39.403338 19891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:39.403350 19891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:39.403363 19891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:39.403374 19891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:39.404861 19891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:46565
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:46565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:39.405102 19891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:39.405341 19891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:39.410146 19896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:39.410166 19897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:39.410280 19891 server_base.cc:1034] running on GCE node
W20260812 14:18:39.410166 19899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:39.410616 19891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:39.410852 19891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:39.411988 19891 hybrid_clock.cc:648] HybridClock initialized: now 1786544319411976 us; error 47 us; skew 500 ppm
I20260812 14:18:39.413151 19891 webserver.cc:460] Webserver started at http://127.8.85.62:46335/ using document root <none> and password file <none>
I20260812 14:18:39.413398 19891 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:39.413460 19891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:39.413553 19891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:39.414688 19891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "68bbd04d94374bb9b8dc2731eef023b1"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "449ca19dcb2d547d88014e0a2fb37b79"
server_key_iv: "a2f147aa912594e99479c5fffa04a0dd"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.415033 19891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "68bbd04d94374bb9b8dc2731eef023b1"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "449ca19dcb2d547d88014e0a2fb37b79"
server_key_iv: "a2f147aa912594e99479c5fffa04a0dd"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.416419 19891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:18:39.417138 19905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.417318 19891 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:18:39.417388 19891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "68bbd04d94374bb9b8dc2731eef023b1"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "449ca19dcb2d547d88014e0a2fb37b79"
server_key_iv: "a2f147aa912594e99479c5fffa04a0dd"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.417483 19891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:39.420852 19891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:39.421134 19891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:39.421274 19891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:39.425433 19891 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:46565
I20260812 14:18:39.425518 19969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:46565 every 8 connection(s)
I20260812 14:18:39.425801 19891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:39.426481 19970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:39.429298 19970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1: Bootstrap starting.
I20260812 14:18:39.429893 19970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:39.430202 19970 log.cc:824] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:39.430883 19970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1: No bootstrap required, opened a new log
I20260812 14:18:39.432575 19970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68bbd04d94374bb9b8dc2731eef023b1" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46565 } }
I20260812 14:18:39.432713 19970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:39.432760 19970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68bbd04d94374bb9b8dc2731eef023b1, State: Initialized, Role: FOLLOWER
I20260812 14:18:39.432909 19970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68bbd04d94374bb9b8dc2731eef023b1" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46565 } }
I20260812 14:18:39.432993 19970 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:39.433035 19970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:39.433082 19970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:39.433614 19970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68bbd04d94374bb9b8dc2731eef023b1" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46565 } }
I20260812 14:18:39.433727 19970 leader_election.cc:302] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68bbd04d94374bb9b8dc2731eef023b1; no voters:
I20260812 14:18:39.433907 19970 leader_election.cc:288] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:39.433960 19974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:39.434149 19974 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [term 1 LEADER]: Becoming Leader. State: Replica: 68bbd04d94374bb9b8dc2731eef023b1, State: Running, Role: LEADER
I20260812 14:18:39.434166 19970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:18:39.434255 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 19891
I20260812 14:18:39.434250 19974 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68bbd04d94374bb9b8dc2731eef023b1" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46565 } }
I20260812 14:18:39.434350 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:39.434610 8532 external_mini_cluster.cc:1428] Setting key 6eb68bb7e1077e57a22b642005995153
I20260812 14:18:39.434602 19974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68bbd04d94374bb9b8dc2731eef023b1. Latest consensus state: current_term: 1 leader_uuid: "68bbd04d94374bb9b8dc2731eef023b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68bbd04d94374bb9b8dc2731eef023b1" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46565 } } }
I20260812 14:18:39.434672 19974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:39.434990 19975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68bbd04d94374bb9b8dc2731eef023b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68bbd04d94374bb9b8dc2731eef023b1" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 46565 } } }
I20260812 14:18:39.435132 19975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1 [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:39.435272 19981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:39.435858 19981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:39.437652 19981 catalog_manager.cc:1347] Generated new cluster ID: ac1816caf4eb4173bdb5bc6454e5495e
I20260812 14:18:39.437651 19986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:39.435093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45414 (local address 127.8.85.62:46565)
0812 14:18:39.435542 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:39.435552 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:39.435585 (+ 33us) server_negotiation.cc:407] Connection header received
0812 14:18:39.435704 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:39.435712 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:39.435756 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:39.435892 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:39.437132 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:39.437136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:39.437136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:39.437141 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:39.437186 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:39.437200 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:39.437275 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:39.437407 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 14:18:39.437473 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":81,"threads_started":1}
I20260812 14:18:39.437798 19981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:18:39.446318 19981 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:39.446502 19981 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:39.460511 19981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68bbd04d94374bb9b8dc2731eef023b1: Generated new TSK 0
I20260812 14:18:39.460716 19981 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:18:39.469740 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:46565
--builtin_ntp_servers=127.8.85.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 14:18:39.560184 19997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:39.560388 19997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:39.560411 19997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:39.561594 19997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:39.561633 19997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:39.561659 19997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:18:39.563035 19997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:46565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:39.563278 19997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:39.563635 19997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:39.569432 20002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:39.569455 20003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:39.569486 20005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:39.569811 19997 server_base.cc:1034] running on GCE node
I20260812 14:18:39.570032 19997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:39.570281 19997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:39.571484 19997 hybrid_clock.cc:648] HybridClock initialized: now 1786544319571470 us; error 28 us; skew 500 ppm
I20260812 14:18:39.572806 19997 webserver.cc:460] Webserver started at http://127.8.85.1:46215/ using document root <none> and password file <none>
I20260812 14:18:39.573060 19997 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:39.573132 19997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:39.573275 19997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:39.574353 19997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "d343613d87384ee6ac0dc3a06125a9b8"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "5717a835b5cd01ac9df1ffd03981da82"
server_key_iv: "45ee5f8b4858cc99edb4e08eeebdd99b"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.574745 19997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "d343613d87384ee6ac0dc3a06125a9b8"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "5717a835b5cd01ac9df1ffd03981da82"
server_key_iv: "45ee5f8b4858cc99edb4e08eeebdd99b"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.576184 19997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:18:39.576911 20011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.577087 19997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:39.577193 19997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "d343613d87384ee6ac0dc3a06125a9b8"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "5717a835b5cd01ac9df1ffd03981da82"
server_key_iv: "45ee5f8b4858cc99edb4e08eeebdd99b"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.577268 19997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:39.580607 19997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:39.580874 19997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:39.581022 19997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:39.581241 19997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:39.581548 19997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:39.581602 19997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.581642 19997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:39.581669 19997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.587221 19997 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:38529
I20260812 14:18:39.587293 20141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:38529 every 8 connection(s)
I20260812 14:18:39.587666 19997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:18:39.590417 19986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:39.588414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:37965 (local address 127.8.85.62:46565)
0812 14:18:39.588497 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:39.588499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:39.588697 (+ 198us) server_negotiation.cc:407] Connection header received
0812 14:18:39.589129 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:39.589130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:39.589137 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:39.589189 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:39.590229 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:39.590230 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:39.590231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:39.590235 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:39.590252 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:39.590256 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:39.590305 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:39.590347 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 14:18:39.590360 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 14:18:39.590605 20144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:39.588379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46565 (local address 127.8.85.1:37965)
0812 14:18:39.588583 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:39.588615 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:39.589049 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:39.589239 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:39.589242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:39.589250 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:39.590089 (+ 839us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:18:39.590090 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:39.590116 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:18:39.590143 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:39.590301 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:39.590303 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:39.590305 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:18:39.590353 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 14:18:39.590398 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":104,"threads_started":1}
I20260812 14:18:39.591065 20142 heartbeater.cc:347] Connected to a master server at 127.8.85.62:46565
I20260812 14:18:39.591151 20142 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:39.591259 20142 heartbeater.cc:511] Master 127.8.85.62:46565 requested a full tablet report, sending...
I20260812 14:18:39.591743 19928 ts_manager.cc:194] Registered new tserver with Master: d343613d87384ee6ac0dc3a06125a9b8 (127.8.85.1:38529)
I20260812 14:18:39.596094 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 19997
I20260812 14:18:39.596172 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:18:39.596328 8532 external_mini_cluster.cc:1428] Setting key 7d3d821f9fe72b86b7dbd5fa13abf0a8
I20260812 14:18:39.597267 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:46565
--builtin_ntp_servers=127.8.85.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 14:18:39.682989 20145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:39.683182 20145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:39.683204 20145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:39.684420 20145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:39.684463 20145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:39.684511 20145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:18:39.685861 20145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:46565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:39.686100 20145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:39.686388 20145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:39.691301 20153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:39.691481 20145 server_base.cc:1034] running on GCE node
W20260812 14:18:39.691321 20151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:39.691439 20155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:39.691821 20145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:39.692073 20145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:39.693265 20145 hybrid_clock.cc:648] HybridClock initialized: now 1786544319693248 us; error 26 us; skew 500 ppm
I20260812 14:18:39.694583 20145 webserver.cc:460] Webserver started at http://127.8.85.2:45807/ using document root <none> and password file <none>
I20260812 14:18:39.694875 20145 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:39.694953 20145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:39.695111 20145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:39.696262 20145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "6c8c45e917f349b78a6b30e066f96502"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "2b35e077ccfa01ad5182f932e3a2139a"
server_key_iv: "d70292cbd764f4051d9a698adc554488"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.696632 20145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "6c8c45e917f349b78a6b30e066f96502"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "2b35e077ccfa01ad5182f932e3a2139a"
server_key_iv: "d70292cbd764f4051d9a698adc554488"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.698055 20145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:39.698822 20161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.699033 20145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:39.699136 20145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "6c8c45e917f349b78a6b30e066f96502"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "2b35e077ccfa01ad5182f932e3a2139a"
server_key_iv: "d70292cbd764f4051d9a698adc554488"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.699222 20145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:39.702495 20145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:39.702837 20145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:39.702968 20145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:39.703195 20145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:39.703574 20145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:39.703630 20145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.703665 20145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:39.703696 20145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.709272 20145 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:34265
I20260812 14:18:39.709355 20290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:34265 every 8 connection(s)
I20260812 14:18:39.709677 20145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:18:39.712618 19986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:39.710416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:39697 (local address 127.8.85.62:46565)
0812 14:18:39.710491 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:39.710493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:39.710727 (+ 234us) server_negotiation.cc:407] Connection header received
0812 14:18:39.711155 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:39.711157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:39.711162 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:39.711220 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:39.712355 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:39.712356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:39.712357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:39.712361 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:39.712378 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:39.712382 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:39.712459 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:39.712552 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 14:18:39.712565 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 14:18:39.712870 20293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:39.710403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46565 (local address 127.8.85.2:39697)
0812 14:18:39.710602 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:39.710640 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:39.711075 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:39.711267 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:39.711270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:39.711278 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:39.712200 (+ 922us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:18:39.712200 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:39.712229 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:18:39.712257 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:39.712473 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:39.712477 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:39.712480 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:18:39.712557 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 14:18:39.712606 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
I20260812 14:18:39.713094 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 20145
I20260812 14:18:39.713219 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:18:39.713464 8532 external_mini_cluster.cc:1428] Setting key 011fca5de6d02b877ba8d318c98839b0
I20260812 14:18:39.713474 20291 heartbeater.cc:347] Connected to a master server at 127.8.85.62:46565
I20260812 14:18:39.713573 20291 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:39.713724 20291 heartbeater.cc:511] Master 127.8.85.62:46565 requested a full tablet report, sending...
I20260812 14:18:39.714102 19927 ts_manager.cc:194] Registered new tserver with Master: 6c8c45e917f349b78a6b30e066f96502 (127.8.85.2:34265)
I20260812 14:18:39.714419 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:46565
--builtin_ntp_servers=127.8.85.20:41115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 14:18:39.796339 20294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:39.796550 20294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:39.796572 20294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:39.797789 20294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:39.797830 20294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:39.797891 20294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:18:39.799324 20294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:46565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:39.799608 20294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:39.799904 20294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:39.804975 20304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:39.805094 20294 server_base.cc:1034] running on GCE node
W20260812 14:18:39.804939 20300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:39.804960 20302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:39.805414 20294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:39.805613 20294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:39.806792 20294 hybrid_clock.cc:648] HybridClock initialized: now 1786544319806784 us; error 35 us; skew 500 ppm
I20260812 14:18:39.808068 20294 webserver.cc:460] Webserver started at http://127.8.85.3:33291/ using document root <none> and password file <none>
I20260812 14:18:39.808264 20294 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:39.808305 20294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:39.808400 20294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:39.809420 20294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "9b53da57d4c7400db08fd309c4440ff6"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "18d9bf4f9841c8853eac4568dae26d12"
server_key_iv: "ea07f9ae4b352a9341aed3f36900f027"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.809727 20294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "9b53da57d4c7400db08fd309c4440ff6"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "18d9bf4f9841c8853eac4568dae26d12"
server_key_iv: "ea07f9ae4b352a9341aed3f36900f027"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.811017 20294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:18:39.811796 20310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.811971 20294 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.812063 20294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "9b53da57d4c7400db08fd309c4440ff6"
format_stamp: "Formatted at 2026-08-12 14:18:39 on dist-test-slave-5k54"
server_key: "18d9bf4f9841c8853eac4568dae26d12"
server_key_iv: "ea07f9ae4b352a9341aed3f36900f027"
server_key_version: "encryptionkey@0"
I20260812 14:18:39.812119 20294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:39.815335 20294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:39.815604 20294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:39.815704 20294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:39.815861 20294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:39.816133 20294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:39.816164 20294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.816182 20294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:39.816195 20294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:39.821898 20294 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:40583
I20260812 14:18:39.821985 20436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:40583 every 8 connection(s)
I20260812 14:18:39.822315 20294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:18:39.825191 19986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:39.823025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:39035 (local address 127.8.85.62:46565)
0812 14:18:39.823093 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:39.823096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:39.823336 (+ 240us) server_negotiation.cc:407] Connection header received
0812 14:18:39.823835 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:39.823836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:39.823842 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:39.823894 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:39.824974 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:39.824975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:39.824976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:18:39.824980 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:39.824995 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:18:39.824999 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:39.825047 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:39.825127 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 14:18:39.825140 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:18:39.825372 20439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:39.823017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:46565 (local address 127.8.85.3:39035)
0812 14:18:39.823223 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:39.823256 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:39.823746 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:39.823948 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:39.823951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:39.823959 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:39.824812 (+ 853us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:18:39.824812 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:39.824840 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:18:39.824868 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:39.825058 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:39.825062 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:39.825064 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:18:39.825123 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 14:18:39.825171 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":107,"threads_started":1}
I20260812 14:18:39.825845 20437 heartbeater.cc:347] Connected to a master server at 127.8.85.62:46565
I20260812 14:18:39.825947 20437 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:39.826054 20437 heartbeater.cc:511] Master 127.8.85.62:46565 requested a full tablet report, sending...
I20260812 14:18:39.826397 19927 ts_manager.cc:194] Registered new tserver with Master: 9b53da57d4c7400db08fd309c4440ff6 (127.8.85.3:40583)
I20260812 14:18:39.829739 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 20294
I20260812 14:18:39.829838 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:18:39.830052 8532 external_mini_cluster.cc:1428] Setting key 32f39565b26be2af14866f42f0c84738
I20260812 14:18:39.830864 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 14:18:39.832705 19986 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:18:39.832215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45416 (local address 127.8.85.62:46565)
0812 14:18:39.832249 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:39.832251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:39.832373 (+ 122us) server_negotiation.cc:407] Connection header received
0812 14:18:39.832459 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:39.832460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:39.832466 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:39.832519 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:39.832642 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45416: BlockingRecv error: recv got EOF from 127.0.0.1:45416 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:18:39.832798 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 19997
I20260812 14:18:39.848415 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 20145
I20260812 14:18:39.855989 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 20294
I20260812 14:18:39.861385 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 19891
2026-08-12T14:18:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (572 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:18:39 dist-test-slave-5k54 krb5kdc[20448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:18:39 dist-test-slave-5k54 krb5kdc[20448](info): set up 2 sockets
Aug 12 14:18:39 dist-test-slave-5k54 krb5kdc[20448](info): commencing operation
krb5kdc: starting...
W20260812 14:18:40.989717 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:18:41 dist-test-slave-5k54 krb5kdc[20448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:18:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:18:41.118974 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:38181
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:43177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:38181
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:41.204885 20464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:41.205116 20464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:41.205155 20464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:41.206278 20464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:18:41.206346 20464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:41.206372 20464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:41.206408 20464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:18:41.206437 20464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:18:41.207971 20464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:43177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:38181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:38181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:41.208244 20464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:41.208469 20464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:41.213326 20475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:41.213353 20472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:41.213352 20470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:41.213631 20464 server_base.cc:1034] running on GCE node
I20260812 14:18:41.213876 20464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:41.214169 20464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:41.215337 20464 hybrid_clock.cc:648] HybridClock initialized: now 1786544321215313 us; error 41 us; skew 500 ppm
Aug 12 14:18:41 dist-test-slave-5k54 krb5kdc[20448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544321, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:41.217788 20464 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:18:41.218196 20464 webserver.cc:460] Webserver started at http://127.8.85.62:46243/ using document root <none> and password file <none>
I20260812 14:18:41.218415 20464 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:41.218492 20464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:41.218639 20464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:41.219779 20464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "cfb4d6f13a1c476c932bd99063574aec"
format_stamp: "Formatted at 2026-08-12 14:18:41 on dist-test-slave-5k54"
server_key: "b9a063f3185a8463f73ae7bc0e9426f3"
server_key_iv: "4208d8b40cf9023e8eb38a7ce2b0088d"
server_key_version: "encryptionkey@0"
I20260812 14:18:41.220136 20464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "cfb4d6f13a1c476c932bd99063574aec"
format_stamp: "Formatted at 2026-08-12 14:18:41 on dist-test-slave-5k54"
server_key: "b9a063f3185a8463f73ae7bc0e9426f3"
server_key_iv: "4208d8b40cf9023e8eb38a7ce2b0088d"
server_key_version: "encryptionkey@0"
I20260812 14:18:41.221544 20464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:41.222447 20485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:41.222642 20464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:41.222754 20464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "cfb4d6f13a1c476c932bd99063574aec"
format_stamp: "Formatted at 2026-08-12 14:18:41 on dist-test-slave-5k54"
server_key: "b9a063f3185a8463f73ae7bc0e9426f3"
server_key_iv: "4208d8b40cf9023e8eb38a7ce2b0088d"
server_key_version: "encryptionkey@0"
I20260812 14:18:41.222827 20464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:41.235724 20464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:41.237494 20464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:41.237653 20464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:41.241771 20464 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:38181
I20260812 14:18:41.241806 20540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:38181 every 8 connection(s)
I20260812 14:18:41.242187 20464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:18:41.243229 20541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:41.244731 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 20464
I20260812 14:18:41.244869 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:18:41.245075 8532 external_mini_cluster.cc:1428] Setting key 938a49d93270ae49dd10cd9624be0cd9
I20260812 14:18:41.246100 20541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec: Bootstrap starting.
I20260812 14:18:41.246803 20541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:41.247149 20541 log.cc:824] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:41.247907 20541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec: No bootstrap required, opened a new log
I20260812 14:18:41.250234 20541 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb4d6f13a1c476c932bd99063574aec" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38181 } }
I20260812 14:18:41.250423 20541 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:41.250468 20541 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfb4d6f13a1c476c932bd99063574aec, State: Initialized, Role: FOLLOWER
I20260812 14:18:41.250610 20541 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb4d6f13a1c476c932bd99063574aec" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38181 } }
I20260812 14:18:41.250715 20541 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:18:41.250751 20541 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:18:41.250805 20541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:41.251607 20541 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb4d6f13a1c476c932bd99063574aec" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38181 } }
I20260812 14:18:41.251735 20541 leader_election.cc:302] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfb4d6f13a1c476c932bd99063574aec; no voters:
I20260812 14:18:41.251906 20541 leader_election.cc:288] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:18:41.251929 20547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:41.252039 20547 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [term 1 LEADER]: Becoming Leader. State: Replica: cfb4d6f13a1c476c932bd99063574aec, State: Running, Role: LEADER
I20260812 14:18:41.252123 20547 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb4d6f13a1c476c932bd99063574aec" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38181 } }
I20260812 14:18:41.252202 20541 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [sys.catalog]: configured and running, proceeding with master startup.
Aug 12 14:18:41 dist-test-slave-5k54 krb5kdc[20448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:41.252444 20547 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfb4d6f13a1c476c932bd99063574aec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb4d6f13a1c476c932bd99063574aec" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38181 } } }
I20260812 14:18:41.252597 20547 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:41.252802 20547 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfb4d6f13a1c476c932bd99063574aec. Latest consensus state: current_term: 1 leader_uuid: "cfb4d6f13a1c476c932bd99063574aec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb4d6f13a1c476c932bd99063574aec" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38181 } } }
I20260812 14:18:41.252949 20547 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec [sys.catalog]: This master's current role is: LEADER
I20260812 14:18:41.253367 20554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:18:41.253893 20554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:18:41.256247 20554 catalog_manager.cc:1347] Generated new cluster ID: 99f342d0f7d74a5f863c7d67817c31e6
I20260812 14:18:41.256294 20554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:18:41.256577 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:41.245640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57030 (local address 127.8.85.62:38181)
0812 14:18:41.245812 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:41.245821 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:41.245855 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:18:41.246329 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:41.246337 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:41.246386 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:41.246505 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:41.247326 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.248371 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.249428 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.249833 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.249908 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:41.252877 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:41.252883 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:41.252884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:41.252898 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:41.254881 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:41.255294 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:41.255296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:41.255297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:41.255375 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:41.255639 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:41.255640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:41.255641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:41.255952 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:41.256122 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:41.256223 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 14:18:41.256364 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260812 14:18:41.276907 20554 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:18:41.277469 20554 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:18:41.281677 20554 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec: Generated new TSK 0
I20260812 14:18:41.281862 20554 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:18:41.309705 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38181
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:43177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:41.393018 20572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:41.393219 20572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:41.393242 20572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:41.394408 20572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:41.394457 20572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:41.394510 20572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:18:41.395956 20572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:43177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:41.396193 20572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:41.396446 20572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:41.401397 20579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:41.401432 20580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:41.401425 20582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:41.401793 20572 server_base.cc:1034] running on GCE node
I20260812 14:18:41.402058 20572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:41.402308 20572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:41.403486 20572 hybrid_clock.cc:648] HybridClock initialized: now 1786544321403469 us; error 40 us; skew 500 ppm
Aug 12 14:18:41 dist-test-slave-5k54 krb5kdc[20448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544321, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:41.406028 20572 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:18:41.406459 20572 webserver.cc:460] Webserver started at http://127.8.85.1:46451/ using document root <none> and password file <none>
I20260812 14:18:41.406693 20572 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:41.406764 20572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:41.406905 20572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:41.408018 20572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "73e5ae66bd7d48bd807bd1f4c67f6093"
format_stamp: "Formatted at 2026-08-12 14:18:41 on dist-test-slave-5k54"
server_key: "228359aa966432f76d552b73c567a356"
server_key_iv: "276d54aa2a9122c509ce173fee8e59df"
server_key_version: "encryptionkey@0"
I20260812 14:18:41.408402 20572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "73e5ae66bd7d48bd807bd1f4c67f6093"
format_stamp: "Formatted at 2026-08-12 14:18:41 on dist-test-slave-5k54"
server_key: "228359aa966432f76d552b73c567a356"
server_key_iv: "276d54aa2a9122c509ce173fee8e59df"
server_key_version: "encryptionkey@0"
I20260812 14:18:41.409770 20572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:41.410526 20592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:41.410704 20572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:41.410794 20572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "73e5ae66bd7d48bd807bd1f4c67f6093"
format_stamp: "Formatted at 2026-08-12 14:18:41 on dist-test-slave-5k54"
server_key: "228359aa966432f76d552b73c567a356"
server_key_iv: "276d54aa2a9122c509ce173fee8e59df"
server_key_version: "encryptionkey@0"
I20260812 14:18:41.410868 20572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:41.432039 20572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:41.433916 20572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:41.434079 20572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:41.434314 20572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:41.434631 20572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:41.434664 20572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:41.434686 20572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:41.434733 20572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:41.440457 20572 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:37835
I20260812 14:18:41.440496 20722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:37835 every 8 connection(s)
I20260812 14:18:41.440822 20572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:18:41.445124 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 20572
I20260812 14:18:41.445236 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:18:41.445447 8532 external_mini_cluster.cc:1428] Setting key 08a97380bc4e18dd477f0159ef4d897c
Aug 12 14:18:41 dist-test-slave-5k54 krb5kdc[20448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544321, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
I20260812 14:18:41.455747 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:41.441538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:40181 (local address 127.8.85.62:38181)
0812 14:18:41.441628 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:41.441631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:41.441845 (+ 214us) server_negotiation.cc:407] Connection header received
0812 14:18:41.442339 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:41.442340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:41.442351 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:41.442403 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:41.443306 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.443863 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.444823 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.445125 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.445198 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:41.452606 (+ 7408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:41.452608 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:41.452608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:41.452616 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:41.454230 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:41.454732 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:41.454733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:41.454733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:41.454788 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:41.455052 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:41.455053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:41.455053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:41.455157 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:41.455207 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:41.455494 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 14:18:41.455608 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 14:18:41.455826 20725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:41.441501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38181 (local address 127.8.85.1:40181)
0812 14:18:41.441686 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:41.441723 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:41.442248 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:41.442465 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:41.442468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:41.443033 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:41.443222 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:41.443227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.443945 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.443946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:41.444732 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:41.444734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.445240 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.445241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:41.445415 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:41.450611 (+ 5196us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:41.450622 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:41.452493 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:41.454316 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:41.454324 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:41.454327 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:41.454649 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:41.454861 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:41.454862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:41.454863 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:41.454992 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:41.455216 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:41.455221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:41.455336 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 14:18:41.455418 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 14:18:41.455634 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":104,"threads_started":1}
I20260812 14:18:41.456285 20723 heartbeater.cc:347] Connected to a master server at 127.8.85.62:38181
I20260812 14:18:41.456393 20723 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:41.456576 20723 heartbeater.cc:511] Master 127.8.85.62:38181 requested a full tablet report, sending...
I20260812 14:18:41.457118 20504 ts_manager.cc:194] Registered new tserver with Master: 73e5ae66bd7d48bd807bd1f4c67f6093 (127.8.85.1:37835)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
I20260812 14:18:41.457890 20504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:40181
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:18:41.474177 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38181
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:43177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:41.558723 20729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:41.558944 20729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:41.558969 20729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:41.560238 20729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:41.560284 20729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:41.560310 20729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:18:41.561781 20729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:43177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:41.562033 20729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:41.562275 20729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:41.567507 20736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:41.567516 20735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:41.567515 20740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:41.567767 20729 server_base.cc:1034] running on GCE node
I20260812 14:18:41.567975 20729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:41.568217 20729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:41.569380 20729 hybrid_clock.cc:648] HybridClock initialized: now 1786544321569369 us; error 44 us; skew 500 ppm
Aug 12 14:18:41 dist-test-slave-5k54 krb5kdc[20448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544321, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:41.572021 20729 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:18:41.572497 20729 webserver.cc:460] Webserver started at http://127.8.85.2:40155/ using document root <none> and password file <none>
I20260812 14:18:41.572742 20729 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:41.572813 20729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:41.572966 20729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:41.574007 20729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "03c54550de634559ba014e74ef09dd51"
format_stamp: "Formatted at 2026-08-12 14:18:41 on dist-test-slave-5k54"
server_key: "5497a3eefdfe9ce1129398aa29bb1b77"
server_key_iv: "307f8ea597e65f9ebc72ef66117325a8"
server_key_version: "encryptionkey@0"
I20260812 14:18:41.574371 20729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "03c54550de634559ba014e74ef09dd51"
format_stamp: "Formatted at 2026-08-12 14:18:41 on dist-test-slave-5k54"
server_key: "5497a3eefdfe9ce1129398aa29bb1b77"
server_key_iv: "307f8ea597e65f9ebc72ef66117325a8"
server_key_version: "encryptionkey@0"
I20260812 14:18:41.575837 20729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:18:41.576565 20747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:41.576731 20729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:18:41.576835 20729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "03c54550de634559ba014e74ef09dd51"
format_stamp: "Formatted at 2026-08-12 14:18:41 on dist-test-slave-5k54"
server_key: "5497a3eefdfe9ce1129398aa29bb1b77"
server_key_iv: "307f8ea597e65f9ebc72ef66117325a8"
server_key_version: "encryptionkey@0"
I20260812 14:18:41.576908 20729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:41.594623 20729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:41.596549 20729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:41.596717 20729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:41.596980 20729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:41.597301 20729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:41.597354 20729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:41.597396 20729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:41.597438 20729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:41.603389 20729 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:42965
I20260812 14:18:41.603479 20878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:42965 every 8 connection(s)
I20260812 14:18:41.603801 20729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:18:41 dist-test-slave-5k54 krb5kdc[20448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544321, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:41.610288 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 20729
I20260812 14:18:41.610412 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:18:41.610651 8532 external_mini_cluster.cc:1428] Setting key 7ebd89c4d7d4b6cb38b9b2800391315d
I20260812 14:18:41.613588 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:41.604522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:45461 (local address 127.8.85.62:38181)
0812 14:18:41.604586 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:41.604588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:41.604823 (+ 235us) server_negotiation.cc:407] Connection header received
0812 14:18:41.605315 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:41.605316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:41.605327 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:41.605386 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:41.606283 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.606840 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.607872 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.608080 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.608139 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:41.610392 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:41.610393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:41.610393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:41.610400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:41.611937 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:41.612441 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:41.612442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:41.612442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:41.612493 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:41.612763 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:41.612763 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:41.612764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:41.612860 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:41.612930 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:41.613291 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 14:18:41.613399 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:18:41.613718 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:41.604496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38181 (local address 127.8.85.2:45461)
0812 14:18:41.604671 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:41.604714 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:41.605229 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:41.605440 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:41.605444 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:41.606008 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:41.606207 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:41.606212 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.606933 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.606934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:41.607794 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:41.607796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.608158 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.608159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:41.608265 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:41.608776 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:41.608784 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:41.610282 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:41.612019 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:41.612027 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:41.612029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:41.612365 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:41.612558 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:41.612559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:41.612559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:41.612701 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:41.612954 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:41.612960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:41.613125 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 14:18:41.613250 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 14:18:41.613434 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260812 14:18:41.614336 20879 heartbeater.cc:347] Connected to a master server at 127.8.85.62:38181
I20260812 14:18:41.614444 20879 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:41.614645 20879 heartbeater.cc:511] Master 127.8.85.62:38181 requested a full tablet report, sending...
I20260812 14:18:41.615036 20504 ts_manager.cc:194] Registered new tserver with Master: 03c54550de634559ba014e74ef09dd51 (127.8.85.2:42965)
I20260812 14:18:41.615577 20504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:45461
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:18:41.637889 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38181
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:43177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:18:41.721493 20885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:18:41.721694 20885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:18:41.721719 20885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:18:41.723003 20885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:18:41.723050 20885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:18:41.723078 20885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:18:41.724604 20885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:43177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:18:41.724879 20885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:18:41.725123 20885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:18:41.730136 20892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:41.730108 20891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:18:41.730129 20896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:18:41.730361 20885 server_base.cc:1034] running on GCE node
I20260812 14:18:41.730564 20885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:18:41.730800 20885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:18:41.731947 20885 hybrid_clock.cc:648] HybridClock initialized: now 1786544321731922 us; error 39 us; skew 500 ppm
Aug 12 14:18:41 dist-test-slave-5k54 krb5kdc[20448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544321, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:18:41.734558 20885 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:18:41.735018 20885 webserver.cc:460] Webserver started at http://127.8.85.3:35261/ using document root <none> and password file <none>
I20260812 14:18:41.735216 20885 fs_manager.cc:359] Metadata directory not provided
I20260812 14:18:41.735256 20885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:18:41.735357 20885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:18:41.736418 20885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "ccc4f01fef3242bba1ce082c37569ac4"
format_stamp: "Formatted at 2026-08-12 14:18:41 on dist-test-slave-5k54"
server_key: "1aa69ba25cbd313992f0dd5e8007e6b8"
server_key_iv: "160bc646471d78dba2faaeb3ecab22c2"
server_key_version: "encryptionkey@0"
I20260812 14:18:41.736726 20885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "ccc4f01fef3242bba1ce082c37569ac4"
format_stamp: "Formatted at 2026-08-12 14:18:41 on dist-test-slave-5k54"
server_key: "1aa69ba25cbd313992f0dd5e8007e6b8"
server_key_iv: "160bc646471d78dba2faaeb3ecab22c2"
server_key_version: "encryptionkey@0"
I20260812 14:18:41.737988 20885 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:18:41.738726 20904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:41.738926 20885 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:18:41.739029 20885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "ccc4f01fef3242bba1ce082c37569ac4"
format_stamp: "Formatted at 2026-08-12 14:18:41 on dist-test-slave-5k54"
server_key: "1aa69ba25cbd313992f0dd5e8007e6b8"
server_key_iv: "160bc646471d78dba2faaeb3ecab22c2"
server_key_version: "encryptionkey@0"
I20260812 14:18:41.739121 20885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:18:41.751924 20885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:18:41.754559 20885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:18:41.754736 20885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:18:41.754980 20885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:18:41.755314 20885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:18:41.755371 20885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:41.755441 20885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:18:41.755473 20885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:18:41.761351 20885 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:44259
I20260812 14:18:41.761389 21030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:44259 every 8 connection(s)
I20260812 14:18:41.761757 20885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:18:41.762832 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 20885
I20260812 14:18:41.762940 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:18:41.763124 8532 external_mini_cluster.cc:1428] Setting key 308cb18876971b13b8daf774aa2dcc92
Aug 12 14:18:41 dist-test-slave-5k54 krb5kdc[20448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544321, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:18:41.771291 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:41.762507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:60167 (local address 127.8.85.62:38181)
0812 14:18:41.762571 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:41.762573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:41.762758 (+ 185us) server_negotiation.cc:407] Connection header received
0812 14:18:41.763502 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:41.763503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:41.763521 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:41.763589 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:41.764698 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.765223 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.766246 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.766451 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.766525 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:41.768619 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:41.768620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:41.768621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:41.768628 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:41.769895 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:41.770299 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:41.770301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:41.770302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:41.770365 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:41.770631 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:41.770633 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:41.770633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:41.770751 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:41.770830 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:41.771118 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 14:18:41.771201 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:18:41.771533 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:41.762496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38181 (local address 127.8.85.3:60167)
0812 14:18:41.762642 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:41.762678 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:41.763304 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:41.763676 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:41.763681 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:41.764392 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:18:41.764590 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:41.764595 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.765328 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.765329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:41.766144 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:41.766146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.766533 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.766534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:41.766676 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:41.767137 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:18:41.767146 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:18:41.768525 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:18:41.769993 (+ 1468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:41.770000 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:41.770002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:41.770222 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:41.770445 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:18:41.770446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:18:41.770447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:18:41.770560 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:18:41.770821 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:18:41.770828 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:18:41.770973 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 14:18:41.771078 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 14:18:41.771236 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260812 14:18:41.771991 21031 heartbeater.cc:347] Connected to a master server at 127.8.85.62:38181
I20260812 14:18:41.772092 21031 heartbeater.cc:464] Registering TS with master...
I20260812 14:18:41.772293 21031 heartbeater.cc:511] Master 127.8.85.62:38181 requested a full tablet report, sending...
I20260812 14:18:41.772675 20504 ts_manager.cc:194] Registered new tserver with Master: ccc4f01fef3242bba1ce082c37569ac4 (127.8.85.3:44259)
I20260812 14:18:41.773080 20504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:60167
I20260812 14:18:41.774727 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:18:41.781847 20545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:41.776057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57034 (local address 127.8.85.62:38181)
0812 14:18:41.776116 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:41.776118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:41.776240 (+ 122us) server_negotiation.cc:407] Connection header received
0812 14:18:41.776279 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:41.776280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:41.776289 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:41.776327 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:18:41.776845 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.777298 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.777969 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.778150 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.778205 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:41.779327 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:18:41.779328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:18:41.779329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:18:41.779336 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:18:41.780631 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:41.780960 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:41.780961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:41.780961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:41.781022 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:18:41.781232 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:18:41.781233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:18:41.781233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:18:41.781386 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:18:41.781500 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:41.781640 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 14:18:41.781741 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:18:41.783214 20504 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:18:41.784839 20504 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:18:41.791771 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:41.787376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:42965 (local address 127.0.0.1:32884)
0812 14:18:41.787745 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:41.787754 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:41.787793 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:41.788325 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:41.788326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:41.788328 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:41.788368 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:41.788370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.789602 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.789604 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:41.790564 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:41.790566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.791410 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.791411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:41.791539 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:41.791559 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 14:18:41.791627 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 14:18:41.791685 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":96,"threads_started":1}
I20260812 14:18:41.791891 21043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:41.787362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:44259 (local address 127.0.0.1:58532)
0812 14:18:41.787577 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:41.787620 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:41.787686 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:41.787991 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:41.787994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:41.787997 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:41.788070 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:41.788072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.789418 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.789419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:41.790797 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:41.790800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.791578 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.791579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:41.791645 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:41.791650 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:18:41.791714 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 14:18:41.791770 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":60,"thread_start_us":109,"threads_started":1}
I20260812 14:18:41.792098 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:41.787622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:37835 (local address 127.0.0.1:43794)
0812 14:18:41.788269 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:41.788280 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:41.788341 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:41.788515 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:41.788516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:41.788518 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:41.788576 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:41.788578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.789451 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.789452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:41.790798 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:41.790800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.791857 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.791858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:41.791925 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:41.791929 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:41.791995 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 14:18:41.792040 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":52,"thread_start_us":75,"threads_started":1}
I20260812 14:18:41.792179 21047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:41.787738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32884 (local address 127.8.85.2:42965)
0812 14:18:41.788097 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:41.788102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:41.788114 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:18:41.788158 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:41.788160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:41.788175 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:41.788250 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:41.788455 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.789500 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.790651 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.791313 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.791392 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:41.791485 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:41.792031 (+ 546us) server_negotiation.cc:299] Negotiation successful
0812 14:18:41.792088 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"spinlock_wait_cycles":10112,"thread_start_us":83,"threads_started":1}
I20260812 14:18:41.792454 21049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:41.788000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43794 (local address 127.8.85.1:37835)
0812 14:18:41.788213 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:41.788217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:41.788338 (+ 121us) server_negotiation.cc:407] Connection header received
0812 14:18:41.788409 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:41.788412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:41.788433 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:41.788519 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:41.788635 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.789314 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.790892 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.791705 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.791767 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:41.791825 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:41.792300 (+ 475us) server_negotiation.cc:299] Negotiation successful
0812 14:18:41.792358 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":89,"threads_started":1}
I20260812 14:18:41.792953 20806 tablet_service.cc:1467] Processing CreateTablet for tablet 0473ceed507441e09fa2a87184e5bb4d (DEFAULT_TABLE table=test-table [id=2673ea914af44df9898a4be14f4a4ff5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:41.793330 20806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0473ceed507441e09fa2a87184e5bb4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:41.795632 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:41.787514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58532 (local address 127.8.85.3:44259)
0812 14:18:41.787786 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:41.787792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:41.787805 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:18:41.787848 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:41.787850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:41.787867 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:41.787968 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:41.788167 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.789303 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.790912 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:41.791490 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:41.791565 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:41.791640 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:41.791741 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 14:18:41.791787 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":111,"threads_started":1}
I20260812 14:18:41.795977 20648 tablet_service.cc:1467] Processing CreateTablet for tablet 0473ceed507441e09fa2a87184e5bb4d (DEFAULT_TABLE table=test-table [id=2673ea914af44df9898a4be14f4a4ff5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:41.796355 20648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0473ceed507441e09fa2a87184e5bb4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:41.796283 20955 tablet_service.cc:1467] Processing CreateTablet for tablet 0473ceed507441e09fa2a87184e5bb4d (DEFAULT_TABLE table=test-table [id=2673ea914af44df9898a4be14f4a4ff5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:18:41.796378 21050 tablet_bootstrap.cc:492] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51: Bootstrap starting.
I20260812 14:18:41.796592 20955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0473ceed507441e09fa2a87184e5bb4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:18:41.797127 21050 tablet_bootstrap.cc:654] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:41.797677 21050 log.cc:824] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:41.798549 21050 tablet_bootstrap.cc:492] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51: No bootstrap required, opened a new log
I20260812 14:18:41.798641 21050 ts_tablet_manager.cc:1397] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:18:41.800004 21052 tablet_bootstrap.cc:492] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4: Bootstrap starting.
I20260812 14:18:41.800705 21052 tablet_bootstrap.cc:654] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:41.800644 21050 raft_consensus.cc:348] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c54550de634559ba014e74ef09dd51" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42965 } } peers { permanent_uuid: "ccc4f01fef3242bba1ce082c37569ac4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44259 } } peers { permanent_uuid: "73e5ae66bd7d48bd807bd1f4c67f6093" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 37835 } }
I20260812 14:18:41.800828 21050 raft_consensus.cc:374] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:41.800884 21050 raft_consensus.cc:729] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03c54550de634559ba014e74ef09dd51, State: Initialized, Role: FOLLOWER
I20260812 14:18:41.801028 21050 consensus_queue.cc:260] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c54550de634559ba014e74ef09dd51" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42965 } } peers { permanent_uuid: "ccc4f01fef3242bba1ce082c37569ac4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44259 } } peers { permanent_uuid: "73e5ae66bd7d48bd807bd1f4c67f6093" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 37835 } }
I20260812 14:18:41.801275 21052 log.cc:824] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:41.801355 21050 ts_tablet_manager.cc:1428] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:18:41.801409 20879 heartbeater.cc:503] Master 127.8.85.62:38181 was elected leader, sending a full tablet report...
I20260812 14:18:41.801574 21053 tablet_bootstrap.cc:492] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093: Bootstrap starting.
I20260812 14:18:41.802126 21052 tablet_bootstrap.cc:492] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4: No bootstrap required, opened a new log
I20260812 14:18:41.802189 21052 ts_tablet_manager.cc:1397] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:18:41.802258 21053 tablet_bootstrap.cc:654] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093: Neither blocks nor log segments found. Creating new log.
I20260812 14:18:41.803246 21053 log.cc:824] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093: Log is configured to *not* fsync() on all Append() calls
I20260812 14:18:41.803941 21053 tablet_bootstrap.cc:492] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093: No bootstrap required, opened a new log
I20260812 14:18:41.803997 21053 ts_tablet_manager.cc:1397] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 14:18:41.804037 21052 raft_consensus.cc:348] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c54550de634559ba014e74ef09dd51" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42965 } } peers { permanent_uuid: "ccc4f01fef3242bba1ce082c37569ac4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44259 } } peers { permanent_uuid: "73e5ae66bd7d48bd807bd1f4c67f6093" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 37835 } }
I20260812 14:18:41.804207 21052 raft_consensus.cc:374] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:41.804251 21052 raft_consensus.cc:729] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccc4f01fef3242bba1ce082c37569ac4, State: Initialized, Role: FOLLOWER
I20260812 14:18:41.804373 21052 consensus_queue.cc:260] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c54550de634559ba014e74ef09dd51" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42965 } } peers { permanent_uuid: "ccc4f01fef3242bba1ce082c37569ac4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44259 } } peers { permanent_uuid: "73e5ae66bd7d48bd807bd1f4c67f6093" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 37835 } }
I20260812 14:18:41.804661 21052 ts_tablet_manager.cc:1428] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:18:41.804666 21031 heartbeater.cc:503] Master 127.8.85.62:38181 was elected leader, sending a full tablet report...
I20260812 14:18:41.805800 21053 raft_consensus.cc:348] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c54550de634559ba014e74ef09dd51" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42965 } } peers { permanent_uuid: "ccc4f01fef3242bba1ce082c37569ac4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44259 } } peers { permanent_uuid: "73e5ae66bd7d48bd807bd1f4c67f6093" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 37835 } }
I20260812 14:18:41.805990 21053 raft_consensus.cc:374] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:18:41.806025 21053 raft_consensus.cc:729] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73e5ae66bd7d48bd807bd1f4c67f6093, State: Initialized, Role: FOLLOWER
I20260812 14:18:41.806131 21053 consensus_queue.cc:260] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c54550de634559ba014e74ef09dd51" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42965 } } peers { permanent_uuid: "ccc4f01fef3242bba1ce082c37569ac4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44259 } } peers { permanent_uuid: "73e5ae66bd7d48bd807bd1f4c67f6093" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 37835 } }
I20260812 14:18:41.806365 21053 ts_tablet_manager.cc:1428] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 14:18:41.806432 20723 heartbeater.cc:503] Master 127.8.85.62:38181 was elected leader, sending a full tablet report...
W20260812 14:18:41.854372 20881 tablet.cc:2369] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:18:41.941720 20724 tablet.cc:2369] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:18:42.012327 21033 tablet.cc:2369] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:18:42.061621 21057 raft_consensus.cc:482] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:18:42.061757 21057 raft_consensus.cc:504] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c54550de634559ba014e74ef09dd51" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42965 } } peers { permanent_uuid: "ccc4f01fef3242bba1ce082c37569ac4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44259 } } peers { permanent_uuid: "73e5ae66bd7d48bd807bd1f4c67f6093" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 37835 } }
I20260812 14:18:42.062175 21057 leader_election.cc:288] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03c54550de634559ba014e74ef09dd51 (127.8.85.2:42965), 73e5ae66bd7d48bd807bd1f4c67f6093 (127.8.85.1:37835)
I20260812 14:18:42.063463 21054 raft_consensus.cc:482] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:18:42.063580 21054 raft_consensus.cc:504] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c54550de634559ba014e74ef09dd51" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42965 } } peers { permanent_uuid: "ccc4f01fef3242bba1ce082c37569ac4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44259 } } peers { permanent_uuid: "73e5ae66bd7d48bd807bd1f4c67f6093" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 37835 } }
I20260812 14:18:42.064555 21054 leader_election.cc:288] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccc4f01fef3242bba1ce082c37569ac4 (127.8.85.3:44259), 73e5ae66bd7d48bd807bd1f4c67f6093 (127.8.85.1:37835)
I20260812 14:18:42.066008 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:42.062406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:42965 (local address 127.8.85.3:42097)
0812 14:18:42.062495 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:42.062524 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:42.062602 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:42.062770 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:42.062771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:42.062773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:42.062864 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:42.062866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.063650 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.063651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:42.064897 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:42.064900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.065723 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.065724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:42.065818 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:42.065822 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:42.065866 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:18:42.065907 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 14:18:42.066287 21047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:42.062482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:42097 (local address 127.8.85.2:42965)
0812 14:18:42.062551 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:42.062553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:42.062579 (+ 26us) server_negotiation.cc:407] Connection header received
0812 14:18:42.062675 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:42.062676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:42.062691 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:42.062757 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:42.062954 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.063549 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.064994 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.065598 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.065673 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:42.065684 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:42.066185 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 14:18:42.066221 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:18:42.066458 21049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:42.062651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:58611 (local address 127.8.85.1:37835)
0812 14:18:42.062724 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:42.062726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:42.062855 (+ 129us) server_negotiation.cc:407] Connection header received
0812 14:18:42.062927 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:42.062928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:42.062945 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:42.063008 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:42.063179 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.063881 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.065341 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.065884 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.065929 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:42.065937 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:42.066337 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 14:18:42.066376 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 14:18:42.066519 20828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0473ceed507441e09fa2a87184e5bb4d" candidate_uuid: "ccc4f01fef3242bba1ce082c37569ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03c54550de634559ba014e74ef09dd51" is_pre_election: true
I20260812 14:18:42.066617 20828 raft_consensus.cc:2454] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccc4f01fef3242bba1ce082c37569ac4 in term 0.
I20260812 14:18:42.066809 20907 leader_election.cc:302] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03c54550de634559ba014e74ef09dd51, ccc4f01fef3242bba1ce082c37569ac4; no voters:
I20260812 14:18:42.066931 21057 raft_consensus.cc:2781] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:18:42.066973 21057 raft_consensus.cc:482] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:18:42.066998 21057 raft_consensus.cc:3037] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:42.067147 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:42.062583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:37835 (local address 127.8.85.3:58611)
0812 14:18:42.062787 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:42.062795 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:42.062860 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:42.063026 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:42.063027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:42.063029 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:42.063110 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:42.063112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.063986 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.063987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:42.065233 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:42.065235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.065999 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.066000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:42.066250 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:42.066254 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:18:42.066940 (+ 686us) client_negotiation.cc:240] Negotiation successful
0812 14:18:42.067058 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":1,"thread_start_us":124,"threads_started":1}
I20260812 14:18:42.067474 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:42.064049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:44259 (local address 127.8.85.2:51999)
0812 14:18:42.064103 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:42.064113 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:42.064147 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:42.064442 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:42.064443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:42.064445 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:42.064499 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:42.064500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.065341 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.065342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:42.066455 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:42.066458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.067212 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.067213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:42.067291 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:42.067294 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:18:42.067338 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:18:42.067378 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 14:18:42.067723 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:42.064237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:51999 (local address 127.8.85.3:44259)
0812 14:18:42.064303 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:42.064305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:42.064313 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:18:42.064333 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:42.064333 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:42.064343 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:42.064400 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:42.064570 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.065257 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.066548 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.067104 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.067169 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:42.067177 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:42.067633 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 14:18:42.067671 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 14:18:42.067977 20976 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0473ceed507441e09fa2a87184e5bb4d" candidate_uuid: "03c54550de634559ba014e74ef09dd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccc4f01fef3242bba1ce082c37569ac4" is_pre_election: true
I20260812 14:18:42.067992 21057 raft_consensus.cc:504] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c54550de634559ba014e74ef09dd51" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42965 } } peers { permanent_uuid: "ccc4f01fef3242bba1ce082c37569ac4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44259 } } peers { permanent_uuid: "73e5ae66bd7d48bd807bd1f4c67f6093" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 37835 } }
I20260812 14:18:42.068123 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:42.064396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:37835 (local address 127.8.85.2:37739)
0812 14:18:42.064669 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:42.064678 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:18:42.064721 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:18:42.065069 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:18:42.065069 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:18:42.065071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:18:42.065133 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:42.065134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.065999 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.065999 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:42.067092 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:18:42.067095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.067876 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.067877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:18:42.067951 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:42.067956 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:18:42.067989 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 14:18:42.068031 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":85,"threads_started":1}
I20260812 14:18:42.068133 20976 raft_consensus.cc:2379] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 03c54550de634559ba014e74ef09dd51 in current term 1: Already voted for candidate ccc4f01fef3242bba1ce082c37569ac4 in this term.
I20260812 14:18:42.068269 20674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0473ceed507441e09fa2a87184e5bb4d" candidate_uuid: "ccc4f01fef3242bba1ce082c37569ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73e5ae66bd7d48bd807bd1f4c67f6093" is_pre_election: true
I20260812 14:18:42.068147 21057 leader_election.cc:288] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [CANDIDATE]: Term 1 election: Requested vote from peers 03c54550de634559ba014e74ef09dd51 (127.8.85.2:42965), 73e5ae66bd7d48bd807bd1f4c67f6093 (127.8.85.1:37835)
I20260812 14:18:42.068380 21062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:42.064821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:37739 (local address 127.8.85.1:37835)
0812 14:18:42.064939 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:42.064940 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:42.064949 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:18:42.064972 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:42.064973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:42.064984 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:42.065192 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:18:42.065223 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.065910 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.067197 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.067771 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.068225 (+ 454us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:42.068239 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:42.068264 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 14:18:42.068316 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260812 14:18:42.068416 20674 raft_consensus.cc:2454] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccc4f01fef3242bba1ce082c37569ac4 in term 0.
I20260812 14:18:42.068684 20828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0473ceed507441e09fa2a87184e5bb4d" candidate_uuid: "ccc4f01fef3242bba1ce082c37569ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03c54550de634559ba014e74ef09dd51"
I20260812 14:18:42.068684 20673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0473ceed507441e09fa2a87184e5bb4d" candidate_uuid: "ccc4f01fef3242bba1ce082c37569ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73e5ae66bd7d48bd807bd1f4c67f6093"
I20260812 14:18:42.068714 20674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0473ceed507441e09fa2a87184e5bb4d" candidate_uuid: "03c54550de634559ba014e74ef09dd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73e5ae66bd7d48bd807bd1f4c67f6093" is_pre_election: true
I20260812 14:18:42.068796 20828 raft_consensus.cc:3037] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:42.068800 20673 raft_consensus.cc:3037] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:18:42.069059 20752 leader_election.cc:302] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 03c54550de634559ba014e74ef09dd51; no voters: 73e5ae66bd7d48bd807bd1f4c67f6093, ccc4f01fef3242bba1ce082c37569ac4
I20260812 14:18:42.069682 20673 raft_consensus.cc:2454] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccc4f01fef3242bba1ce082c37569ac4 in term 1.
I20260812 14:18:42.069707 20828 raft_consensus.cc:2454] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccc4f01fef3242bba1ce082c37569ac4 in term 1.
I20260812 14:18:42.069801 21054 raft_consensus.cc:2726] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 14:18:42.069864 20907 leader_election.cc:302] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73e5ae66bd7d48bd807bd1f4c67f6093, ccc4f01fef3242bba1ce082c37569ac4; no voters:
I20260812 14:18:42.069955 21057 raft_consensus.cc:2781] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:18:42.070106 21057 raft_consensus.cc:686] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [term 1 LEADER]: Becoming Leader. State: Replica: ccc4f01fef3242bba1ce082c37569ac4, State: Running, Role: LEADER
I20260812 14:18:42.070197 21057 consensus_queue.cc:237] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c54550de634559ba014e74ef09dd51" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42965 } } peers { permanent_uuid: "ccc4f01fef3242bba1ce082c37569ac4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44259 } } peers { permanent_uuid: "73e5ae66bd7d48bd807bd1f4c67f6093" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 37835 } }
I20260812 14:18:42.071002 20504 catalog_manager.cc:5495] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccc4f01fef3242bba1ce082c37569ac4 (127.8.85.3). New cstate: current_term: 1 leader_uuid: "ccc4f01fef3242bba1ce082c37569ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03c54550de634559ba014e74ef09dd51" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccc4f01fef3242bba1ce082c37569ac4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73e5ae66bd7d48bd807bd1f4c67f6093" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 37835 } health_report { overall_health: UNKNOWN } } }
I20260812 14:18:42.078487 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:42.076235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58546 (local address 127.8.85.3:44259)
0812 14:18:42.076269 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 14:18:42.076271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:18:42.076288 (+ 17us) server_negotiation.cc:407] Connection header received
0812 14:18:42.076361 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:18:42.076361 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:18:42.076375 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:18:42.076435 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:18:42.076712 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.077134 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.077880 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:18:42.078051 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:18:42.078099 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:18:42.078230 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:18:42.078320 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:18:42.078378 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:18:42.078411 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 14:18:42.078426 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 14:18:42.080292 20828 raft_consensus.cc:1261] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51 [term 1 FOLLOWER]: Refusing update from remote peer ccc4f01fef3242bba1ce082c37569ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:42.080313 20673 raft_consensus.cc:1261] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093 [term 1 FOLLOWER]: Refusing update from remote peer ccc4f01fef3242bba1ce082c37569ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:18:42.080626 21063 consensus_queue.cc:1035] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73e5ae66bd7d48bd807bd1f4c67f6093" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 37835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:42.080742 21057 consensus_queue.cc:1035] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03c54550de634559ba014e74ef09dd51" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:18:42.083108 21066 mvcc.cc:205] Tried to move back new op lower bound from 7317685543238078464 to 7317685543200010240. Current Snapshot: MvccSnapshot[applied={T|T < 7317685543238078464}]
I20260812 14:18:42.083952 21067 mvcc.cc:205] Tried to move back new op lower bound from 7317685543238078464 to 7317685543200010240. Current Snapshot: MvccSnapshot[applied={T|T < 7317685543238078464}]
I20260812 14:18:42.085996 20503 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:42.086105 20503 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:18:42.086861 20503 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfb4d6f13a1c476c932bd99063574aec: Sending DeleteTablet for 3 replicas of tablet 0473ceed507441e09fa2a87184e5bb4d
I20260812 14:18:42.087218 20955 tablet_service.cc:1514] Processing DeleteTablet for tablet 0473ceed507441e09fa2a87184e5bb4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:42 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:58532
I20260812 14:18:42.087224 20806 tablet_service.cc:1514] Processing DeleteTablet for tablet 0473ceed507441e09fa2a87184e5bb4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:42 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:32884
I20260812 14:18:42.087482 21079 tablet_replica.cc:330] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51: stopping tablet replica
I20260812 14:18:42.087517 20648 tablet_service.cc:1514] Processing DeleteTablet for tablet 0473ceed507441e09fa2a87184e5bb4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:18:42 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:43794
I20260812 14:18:42.087585 21079 raft_consensus.cc:2229] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:42.087685 21079 raft_consensus.cc:2258] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:42.087719 21080 tablet_replica.cc:330] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093: stopping tablet replica
I20260812 14:18:42.087723 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 20572
I20260812 14:18:42.087809 21080 raft_consensus.cc:2229] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:18:42.087908 21080 raft_consensus.cc:2258] T 0473ceed507441e09fa2a87184e5bb4d P 73e5ae66bd7d48bd807bd1f4c67f6093 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:42.088143 21079 ts_tablet_manager.cc:1905] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:42.090663 21079 ts_tablet_manager.cc:1918] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:42.090754 21079 log.cc:1196] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/0473ceed507441e09fa2a87184e5bb4d
I20260812 14:18:42.091118 21079 ts_tablet_manager.cc:1939] T 0473ceed507441e09fa2a87184e5bb4d P 03c54550de634559ba014e74ef09dd51: Deleting consensus metadata
I20260812 14:18:42.091610 21078 tablet_replica.cc:330] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4: stopping tablet replica
I20260812 14:18:42.091732 21078 raft_consensus.cc:2229] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:18:42.092111 21078 raft_consensus.cc:2258] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:18:42.094635 20488 catalog_manager.cc:4841] TS 03c54550de634559ba014e74ef09dd51 (127.8.85.2:42965): tablet 0473ceed507441e09fa2a87184e5bb4d (table test-table [id=2673ea914af44df9898a4be14f4a4ff5]) successfully deleted
I20260812 14:18:42.094949 21078 ts_tablet_manager.cc:1905] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:18:42.096776 21078 ts_tablet_manager.cc:1918] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:18:42.096853 21078 log.cc:1196] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/0473ceed507441e09fa2a87184e5bb4d
I20260812 14:18:42.097164 21078 ts_tablet_manager.cc:1939] T 0473ceed507441e09fa2a87184e5bb4d P ccc4f01fef3242bba1ce082c37569ac4: Deleting consensus metadata
I20260812 14:18:42.097592 20489 catalog_manager.cc:4841] TS ccc4f01fef3242bba1ce082c37569ac4 (127.8.85.3:44259): tablet 0473ceed507441e09fa2a87184e5bb4d (table test-table [id=2673ea914af44df9898a4be14f4a4ff5]) successfully deleted
W20260812 14:18:42.099076 20488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:37835 (error 108)
I20260812 14:18:42.099383 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 20729
I20260812 14:18:42.100720 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:18:42.099522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:37835 (local address 127.0.0.1:43804)
0812 14:18:42.100640 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0812 14:18:42.100682 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:37835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1093}
W20260812 14:18:42.101567 20488 catalog_manager.cc:4584] TS 73e5ae66bd7d48bd807bd1f4c67f6093 (127.8.85.1:37835): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0473ceed507441e09fa2a87184e5bb4d: Network error: Client connection negotiation failed: client connection to 127.8.85.1:37835: connect: Connection refused (error 111)
I20260812 14:18:42.109234 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 20885
I20260812 14:18:42.115413 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 20464
2026-08-12T14:18:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2253 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:18:42 dist-test-slave-5k54 krb5kdc[21084](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:18:42 dist-test-slave-5k54 krb5kdc[21084](info): set up 2 sockets
Aug 12 14:18:42 dist-test-slave-5k54 krb5kdc[21084](info): commencing operation
krb5kdc: starting...
W20260812 14:18:43.244527 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:18:43 dist-test-slave-5k54 krb5kdc[21084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:18:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:18:43Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.85.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.85.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/rangeradmin_127.8.85.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/rangeradmin_127.8.85.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.85.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.85.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/rangerlookup_127.8.85.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/rangerlookup_127.8.85.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.85.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.85.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/HTTP_127.8.85.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/HTTP_127.8.85.20@KRBTEST.COM.keytab.
I20260812 14:18:43.417796 8532 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskYVQ8wa/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/postgres -l logfile start
2026-08-12 14:18:45.928 UTC [21121] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 14:18:45.928 UTC [21121] LOG: listening on IPv4 address "127.8.85.20", port 47023
2026-08-12 14:18:45.932 UTC [21121] LOG: listening on Unix socket "/tmp/.s.PGSQL.47023"
2026-08-12 14:18:45.967 UTC [21124] LOG: database system was shut down at 2026-08-12 14:18:44 UTC
2026-08-12 14:18:45.972 UTC [21121] LOG: database system is ready to accept connections
I20260812 14:18:46.990936 8532 mini_postgres.cc:96] Postgres bound to 47023
2026-08-12 14:18:46.999 UTC [21132] FATAL: database "slave" does not exist
127.8.85.20:47023 - accepting connections
I20260812 14:18:47.000007 8532 mini_ranger.cc:161] Starting Ranger...
I20260812 14:18:47.017151 8532 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 14:18:47.507869 8532 mini_ranger.cc:87] Created ranger Postgres database
I20260812 14:18:47.507953 8532 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-admin
2026-08-12 14:18:47,843 [I] DB FLAVOR :POSTGRES
2026-08-12 14:18:47,843 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 14:18:47,843 [I] Checking connection..
2026-08-12 14:18:47,843 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 14:18:48,113 [I] Checking connection passed.
2026-08-12 14:18:48,113 [I] --------- Verifying version history table ---------
2026-08-12 14:18:48,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:18:48,354 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 14:18:48,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:18:48,609 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 14:18:48,609 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 14:18:48,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 14:18:48.884 UTC [21207] WARNING: there is no transaction in progress
2026-08-12 14:18:48,893 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 14:18:48,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:18:49,154 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 14:18:49,154 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 14:18:49,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 14:18:49,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 14:18:49,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5k54.c.gcp-upstream.internal','N') ;"
2026-08-12 14:18:49,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 14:18:50,133 [I] Table x_portal_user does not exist in database ranger
2026-08-12 14:18:50,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 14:18:50,398 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 14:18:50,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 14:18:50,657 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 14:18:50,920 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 14:18:50,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 14:18:51.862 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:51.877 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:51.893 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:51.907 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:51.925 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:51.944 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:51.997 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:52.004 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:52.018 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:52.028 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:52.039 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:52.053 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:52.062 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:52.068 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:52.075 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:52.082 UTC [21330] WARNING: there is no transaction in progress
2026-08-12 14:18:52,700 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 14:18:52,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 14:18:52,953 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 14:18:52,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 14:18:53,218 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 14:18:53,219 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 14:18:53,459 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 14:18:53,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 14:18:53,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 14:18:53,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5k54.c.gcp-upstream.internal';"
2026-08-12 14:18:54,253 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 14:18:54,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 14:18:54,490 [I] DB_PATCHES have already been applied
I20260812 14:18:54.496667 8532 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-admin:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskYVQ8wa/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 14:18:54.496758 8532 mini_ranger.cc:193] Using host: 127.8.85.20
I20260812 14:18:54.500720 8532 mini_ranger.cc:227] Ranger admin URL: http://127.8.85.20:54905
Aug 12, 2026 2:18:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:18:54 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:18:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 2:18:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 2:18:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 2:18:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 2:18:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 14:18:55.738740 8532 mini_ranger.cc:160] Time spent starting Ranger: real 8.739s user 0.000s sys 0.011s
Aug 12, 2026 2:19:15 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 2:19:15 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 12, 2026 2:19:15 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 2:19:15 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 2:19:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 2:19:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 2:19:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 2:19:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 2:19:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 2:19:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 2:19:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 2:19:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 2:19:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 2:19:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 2:19:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 2:19:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 2:19:15 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 2:19:16 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 14:19:16.610314 8532 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.85.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.85.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/rangerkms_127.8.85.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/rangerkms_127.8.85.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.85.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.85.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.85.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/HTTP_127.8.85.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.85.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/HTTP_127.8.85.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 14:19:16 dist-test-slave-5k54 krb5kdc[21084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544356, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 14:19:16.692929 8532 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 14:19:16.703075 8532 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 14:19:17.537839 8532 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 14:19:17.537919 8532 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-kms
I20260812 14:19:17.537935 8532 mini_ranger_kms.cc:227] Using postgres at 127.8.85.20:47023
2026-08-12 14:19:17,977 [I] DB FLAVOR :POSTGRES
2026-08-12 14:19:17,978 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 14:19:17,978 [I] Checking connection
2026-08-12 14:19:17,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 14:19:18,226 [I] connection success
2026-08-12 14:19:18,226 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 14:19:18,226 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 14:19:18,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 14:19:18,466 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 14:19:18,466 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 14:19:18,467 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 14:19:18,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:47023/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 14:19:18,731 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 14:19:19.217690 8532 mini_ranger_kms.cc:310] Created kms service
I20260812 14:19:19.405647 8532 mini_ranger_kms.cc:326] Created kudu user
I20260812 14:19:19.449890 8532 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 14:19:19 dist-test-slave-5k54 krb5kdc[21084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544356, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:19:19.851624 8532 mini_ranger_kms.cc:384] Added ranger policy
I20260812 14:19:19.851922 8532 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-kms:/tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 14:19:19.851994 8532 mini_ranger_kms.cc:240] Using host: 127.8.85.20
I20260812 14:19:19.856688 8532 mini_ranger_kms.cc:276] Ranger KMS PID: 21536
I20260812 14:19:19.856802 8532 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.85.20:37087
Aug 12, 2026 2:19:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:19:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:19:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 2:19:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 2:19:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 2:19:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 14:19:22 dist-test-slave-5k54 krb5kdc[21084](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786544362, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.85.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:19:22.664925 8532 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.972s user 0.006s sys 0.250s
I20260812 14:19:22.665040 8532 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 14:19:22.665118 8532 mini_ranger_kms.cc:401] 127.8.85.20:37087/kms/v1/keys
Aug 12 14:19:23 dist-test-slave-5k54 krb5kdc[21084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544362, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.85.20@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
Aug 12 14:19:23 dist-test-slave-5k54 krb5kdc[21084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544362, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.85.20@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
Aug 12 14:19:25 dist-test-slave-5k54 krb5kdc[21084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:19:25.770394 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:38199
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:33351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:38199
--ranger_config_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.85.20:37087/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:19:25.855471 21593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:19:25.855707 21593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:19:25.855746 21593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:19:25.856910 21593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:19:25.856976 21593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:19:25.857002 21593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:19:25.857031 21593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:19:25.857052 21593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:19:25.858666 21593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:33351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.85.20:37087/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:38199
--ranger_config_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:38199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:19:25.858932 21593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:19:25.859143 21593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:19:25.864454 21602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:19:25.864472 21601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:19:25.864500 21593 server_base.cc:1034] running on GCE node
W20260812 14:19:25.864472 21605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:19:25.864964 21593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:19:25.865247 21593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:19:25.866454 21593 hybrid_clock.cc:648] HybridClock initialized: now 1786544365866440 us; error 26 us; skew 500 ppm
Aug 12 14:19:25 dist-test-slave-5k54 krb5kdc[21084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544365, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:19:25.868973 21593 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:19:25.869474 21593 webserver.cc:460] Webserver started at http://127.8.85.62:38759/ using document root <none> and password file <none>
I20260812 14:19:25.869693 21593 fs_manager.cc:359] Metadata directory not provided
I20260812 14:19:25.869748 21593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:19:25.869896 21593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:19:25 dist-test-slave-5k54 krb5kdc[21084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544365, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:19:25.931867 21593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "6416ca4b6cc44b5e879b50688aa0ca19"
format_stamp: "Formatted at 2026-08-12 14:19:25 on dist-test-slave-5k54"
server_key: "cab2debd067ff6081f1da740c93c8388"
server_key_iv: "9ec168522d08ac19ee0da333dbde3ebb"
server_key_version: "kuduclusterkey@0"
I20260812 14:19:25.932361 21593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "6416ca4b6cc44b5e879b50688aa0ca19"
format_stamp: "Formatted at 2026-08-12 14:19:25 on dist-test-slave-5k54"
server_key: "cab2debd067ff6081f1da740c93c8388"
server_key_iv: "9ec168522d08ac19ee0da333dbde3ebb"
server_key_version: "kuduclusterkey@0"
I20260812 14:19:25.934284 21593 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 14:19:25.949103 21614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:19:25.949326 21593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:19:25.949424 21593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "6416ca4b6cc44b5e879b50688aa0ca19"
format_stamp: "Formatted at 2026-08-12 14:19:25 on dist-test-slave-5k54"
server_key: "cab2debd067ff6081f1da740c93c8388"
server_key_iv: "9ec168522d08ac19ee0da333dbde3ebb"
server_key_version: "kuduclusterkey@0"
I20260812 14:19:25.949522 21593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:19:25.974409 21593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:19:25.976573 21593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:19:25.976742 21593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:19:25.981545 21593 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:38199
I20260812 14:19:25.981616 21669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:38199 every 8 connection(s)
I20260812 14:19:25.982043 21593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:19:25.982754 21670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:19:25.985159 21670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19: Bootstrap starting.
I20260812 14:19:25.985846 21670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19: Neither blocks nor log segments found. Creating new log.
I20260812 14:19:25.986123 21670 log.cc:824] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19: Log is configured to *not* fsync() on all Append() calls
I20260812 14:19:25.986865 21670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19: No bootstrap required, opened a new log
I20260812 14:19:25.987155 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 21593
I20260812 14:19:25.987262 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:19:25.987466 8532 external_mini_cluster.cc:1428] Setting key e098f4972c55dc2235378d6ae316a9a2
I20260812 14:19:25.989818 21670 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6416ca4b6cc44b5e879b50688aa0ca19" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38199 } }
I20260812 14:19:25.990068 21670 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:19:25.990144 21670 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6416ca4b6cc44b5e879b50688aa0ca19, State: Initialized, Role: FOLLOWER
I20260812 14:19:25.990338 21670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6416ca4b6cc44b5e879b50688aa0ca19" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38199 } }
I20260812 14:19:25.990437 21670 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:19:25.990494 21670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:19:25.990569 21670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:19:25.991294 21670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6416ca4b6cc44b5e879b50688aa0ca19" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38199 } }
I20260812 14:19:25.991410 21670 leader_election.cc:302] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6416ca4b6cc44b5e879b50688aa0ca19; no voters:
I20260812 14:19:25.991638 21670 leader_election.cc:288] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:19:25.991695 21677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:19:25.991895 21677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [term 1 LEADER]: Becoming Leader. State: Replica: 6416ca4b6cc44b5e879b50688aa0ca19, State: Running, Role: LEADER
I20260812 14:19:25.991981 21670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:19:25.992051 21677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6416ca4b6cc44b5e879b50688aa0ca19" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38199 } }
I20260812 14:19:25.993549 21678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6416ca4b6cc44b5e879b50688aa0ca19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6416ca4b6cc44b5e879b50688aa0ca19" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38199 } } }
I20260812 14:19:25.993705 21678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [sys.catalog]: This master's current role is: LEADER
I20260812 14:19:25.993881 21678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6416ca4b6cc44b5e879b50688aa0ca19. Latest consensus state: current_term: 1 leader_uuid: "6416ca4b6cc44b5e879b50688aa0ca19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6416ca4b6cc44b5e879b50688aa0ca19" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38199 } } }
I20260812 14:19:25.993993 21678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19 [sys.catalog]: This master's current role is: LEADER
I20260812 14:19:25.994041 21670 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 14:19:25.994204 21692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:19:25.997483 21692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 14:19:25 dist-test-slave-5k54 krb5kdc[21084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:19:26.000962 21692 catalog_manager.cc:1347] Generated new cluster ID: 2bc2a90e66144375a3e962ff06bd94c9
I20260812 14:19:26.001063 21692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:19:26.003458 21675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:25.987992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50022 (local address 127.8.85.62:38199)
0812 14:19:25.988172 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 14:19:25.988181 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:19:25.988212 (+ 31us) server_negotiation.cc:407] Connection header received
0812 14:19:25.989757 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:19:25.989777 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:19:25.989843 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:19:25.989998 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:19:25.990774 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:25.992257 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:25.993359 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:25.993964 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:25.994076 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:25.998515 (+ 4439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:19:25.998557 (+ 42us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:19:25.998558 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:19:25.998636 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:19:26.001430 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:19:26.001811 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:19:26.001814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:19:26.001814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:19:26.001945 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:19:26.002139 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:19:26.002140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:19:26.002140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:19:26.002489 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:19:26.002645 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 14:19:26.002762 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 14:19:26.003166 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":78,"threads_started":1}
I20260812 14:19:26.013921 21692 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:19:26.014663 21692 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:19:26.018882 21692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19: Generated new TSK 0
I20260812 14:19:26.019052 21692 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:19:26.762671 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.85.20:37087/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38199
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:33351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:19:26.888329 21726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:19:26.888645 21726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:19:26.888685 21726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:19:26.890372 21726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:19:26.890465 21726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:19:26.890514 21726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:19:26.892689 21726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:33351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.85.20:37087/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:19:26.892971 21726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:19:26.893262 21726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:19:26.899727 21732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:19:26.899736 21733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:19:26.900043 21735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:19:26.900411 21726 server_base.cc:1034] running on GCE node
I20260812 14:19:26.900619 21726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:19:26.900918 21726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:19:26.902102 21726 hybrid_clock.cc:648] HybridClock initialized: now 1786544366902082 us; error 30 us; skew 500 ppm
Aug 12 14:19:26 dist-test-slave-5k54 krb5kdc[21084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:19:26.905328 21726 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:19:26.905886 21726 webserver.cc:460] Webserver started at http://127.8.85.1:42009/ using document root <none> and password file <none>
I20260812 14:19:26.906198 21726 fs_manager.cc:359] Metadata directory not provided
I20260812 14:19:26.906270 21726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:19:26.906458 21726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:19:26 dist-test-slave-5k54 krb5kdc[21084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:19:26.921777 21726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "46e0fa74015c4973a9b6cd9a737d37fa"
format_stamp: "Formatted at 2026-08-12 14:19:26 on dist-test-slave-5k54"
server_key: "2c646bcff9056eeaf83565a908c890b3"
server_key_iv: "f2e5a6ac9f88ac0cf00349272d2ba799"
server_key_version: "kuduclusterkey@0"
I20260812 14:19:26.922268 21726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "46e0fa74015c4973a9b6cd9a737d37fa"
format_stamp: "Formatted at 2026-08-12 14:19:26 on dist-test-slave-5k54"
server_key: "2c646bcff9056eeaf83565a908c890b3"
server_key_iv: "f2e5a6ac9f88ac0cf00349272d2ba799"
server_key_version: "kuduclusterkey@0"
I20260812 14:19:26.925215 21726 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 14:19:26.941664 21744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:19:26.941916 21726 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 14:19:26.942034 21726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "46e0fa74015c4973a9b6cd9a737d37fa"
format_stamp: "Formatted at 2026-08-12 14:19:26 on dist-test-slave-5k54"
server_key: "2c646bcff9056eeaf83565a908c890b3"
server_key_iv: "f2e5a6ac9f88ac0cf00349272d2ba799"
server_key_version: "kuduclusterkey@0"
I20260812 14:19:26.942143 21726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:19:26.964231 21726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:19:26.966604 21726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:19:26.966771 21726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:19:26.967006 21726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:19:26.967401 21726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:19:26.967480 21726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:19:26.967548 21726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:19:26.967592 21726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:19:26.975358 21726 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:33927
I20260812 14:19:26.975905 21726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:19:26.976506 21871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:33927 every 8 connection(s)
I20260812 14:19:26.984282 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 21726
I20260812 14:19:26.984387 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:19:26.984588 8532 external_mini_cluster.cc:1428] Setting key 064e41e5d32f44c0d21f4f8322e2ba99
Aug 12 14:19:26 dist-test-slave-5k54 krb5kdc[21084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544366, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:19:26.988893 21874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:26.977417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38199 (local address 127.8.85.1:45527)
0812 14:19:26.977999 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0812 14:19:26.978048 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 14:19:26.978693 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:19:26.979158 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:19:26.979163 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:19:26.979406 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:19:26.979739 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:26.979745 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:26.980809 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:26.980810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:26.982005 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:26.982007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:26.982540 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:26.982540 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:26.982696 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:26.983304 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:19:26.983317 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:19:26.985053 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:19:26.987228 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:19:26.987236 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:19:26.987238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:19:26.987599 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:19:26.987807 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:19:26.987808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:19:26.987808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:19:26.987938 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:19:26.988219 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:19:26.988225 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:19:26.988396 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 14:19:26.988508 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 14:19:26.988662 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":84,"threads_started":1}
I20260812 14:19:26.989460 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:26.977729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:45527 (local address 127.8.85.62:38199)
0812 14:19:26.978883 (+ 1154us) server_negotiation.cc:206] Beginning negotiation
0812 14:19:26.978885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:19:26.978894 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:19:26.978920 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:19:26.978921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:19:26.978934 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:19:26.978996 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:19:26.979853 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:26.980701 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:26.982098 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:26.982402 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:26.982472 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:26.985147 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:19:26.985149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:19:26.985149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:19:26.985156 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:19:26.987134 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:19:26.987681 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:19:26.987682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:19:26.987683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:19:26.987744 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:19:26.988006 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:19:26.988007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:19:26.988007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:19:26.988140 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:19:26.988196 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:19:26.989268 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0812 14:19:26.989352 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1121,"thread_start_us":76,"threads_started":1}
I20260812 14:19:26.991698 21872 heartbeater.cc:347] Connected to a master server at 127.8.85.62:38199
I20260812 14:19:26.991793 21872 heartbeater.cc:464] Registering TS with master...
I20260812 14:19:26.991986 21872 heartbeater.cc:511] Master 127.8.85.62:38199 requested a full tablet report, sending...
I20260812 14:19:26.992565 21632 ts_manager.cc:194] Registered new tserver with Master: 46e0fa74015c4973a9b6cd9a737d37fa (127.8.85.1:33927)
I20260812 14:19:26.993432 21632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:45527
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:19:27.023329 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.85.20:37087/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38199
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:33351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:19:27.134920 21880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:19:27.135223 21880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:19:27.135268 21880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:19:27.136528 21880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:19:27.136600 21880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:19:27.136646 21880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:19:27.138324 21880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:33351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.85.20:37087/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:19:27.138598 21880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:19:27.138849 21880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:19:27.144169 21888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:19:27.144143 21890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:19:27.144459 21887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:19:27.145179 21880 server_base.cc:1034] running on GCE node
I20260812 14:19:27.145394 21880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:19:27.145679 21880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:19:27.146832 21880 hybrid_clock.cc:648] HybridClock initialized: now 1786544367146814 us; error 39 us; skew 500 ppm
Aug 12 14:19:27 dist-test-slave-5k54 krb5kdc[21084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:19:27.149897 21880 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:19:27.150394 21880 webserver.cc:460] Webserver started at http://127.8.85.2:33343/ using document root <none> and password file <none>
I20260812 14:19:27.150655 21880 fs_manager.cc:359] Metadata directory not provided
I20260812 14:19:27.150727 21880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:19:27.150892 21880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:19:27 dist-test-slave-5k54 krb5kdc[21084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:19:27.165637 21880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "6a44cfb795334c80955ceedba22cf754"
format_stamp: "Formatted at 2026-08-12 14:19:27 on dist-test-slave-5k54"
server_key: "887c30102358ae83f104efd7a6e75fb9"
server_key_iv: "6bd15ffb12e4381e6fdcd8f72553a8f1"
server_key_version: "kuduclusterkey@0"
I20260812 14:19:27.166200 21880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "6a44cfb795334c80955ceedba22cf754"
format_stamp: "Formatted at 2026-08-12 14:19:27 on dist-test-slave-5k54"
server_key: "887c30102358ae83f104efd7a6e75fb9"
server_key_iv: "6bd15ffb12e4381e6fdcd8f72553a8f1"
server_key_version: "kuduclusterkey@0"
Aug 12 14:19:27 dist-test-slave-5k54 krb5kdc[21084](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786544367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:19:27.169977 21880 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 14:19:27.189124 21899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:19:27.192498 21880 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260812 14:19:27.192637 21880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "6a44cfb795334c80955ceedba22cf754"
format_stamp: "Formatted at 2026-08-12 14:19:27 on dist-test-slave-5k54"
server_key: "887c30102358ae83f104efd7a6e75fb9"
server_key_iv: "6bd15ffb12e4381e6fdcd8f72553a8f1"
server_key_version: "kuduclusterkey@0"
I20260812 14:19:27.192777 21880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:19:27.214829 21880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:19:27.217552 21880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:19:27.217773 21880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:19:27.218102 21880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:19:27.218506 21880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:19:27.218591 21880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:19:27.218662 21880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:19:27.218719 21880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:19:27.226480 21880 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:42385
I20260812 14:19:27.226962 21880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:19:27.227748 22023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:42385 every 8 connection(s)
Aug 12 14:19:27 dist-test-slave-5k54 krb5kdc[21084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:19:27.236459 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 21880
I20260812 14:19:27.236563 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:19:27.236780 8532 external_mini_cluster.cc:1428] Setting key a2561a3a097284a9db2ec5fd8ccd7593
I20260812 14:19:27.240458 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.228720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:60081 (local address 127.8.85.62:38199)
0812 14:19:27.228775 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:19:27.228777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:19:27.228915 (+ 138us) server_negotiation.cc:407] Connection header received
0812 14:19:27.229603 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:19:27.229604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:19:27.229617 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:19:27.229650 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:19:27.230472 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.231273 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.232787 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.233119 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.233159 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.235937 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:19:27.235939 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:19:27.235939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:19:27.235949 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:19:27.237870 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:19:27.239209 (+ 1339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:19:27.239211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:19:27.239211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:19:27.239277 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:19:27.239612 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:19:27.239613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:19:27.239613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:19:27.239756 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:19:27.239798 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 14:19:27.240250 (+ 452us) server_negotiation.cc:299] Negotiation successful
0812 14:19:27.240352 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:19:27.240671 22026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.228462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38199 (local address 127.8.85.2:60081)
0812 14:19:27.228775 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 14:19:27.228825 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 14:19:27.229485 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:19:27.229831 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:19:27.229835 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:19:27.230083 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:19:27.230353 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:27.230360 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.231380 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.231381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:27.232682 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:27.232685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.233245 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.233246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:27.233392 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.234015 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:19:27.234026 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:19:27.235819 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:19:27.237967 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:19:27.237975 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:19:27.237978 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:19:27.239114 (+ 1136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:19:27.239346 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:19:27.239347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:19:27.239347 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:19:27.239529 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:19:27.239845 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:19:27.239850 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:19:27.240050 (+ 200us) client_negotiation.cc:769] Sending connection context
0812 14:19:27.240192 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 14:19:27.240366 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":96,"threads_started":1}
I20260812 14:19:27.241273 22024 heartbeater.cc:347] Connected to a master server at 127.8.85.62:38199
I20260812 14:19:27.241379 22024 heartbeater.cc:464] Registering TS with master...
I20260812 14:19:27.241593 22024 heartbeater.cc:511] Master 127.8.85.62:38199 requested a full tablet report, sending...
I20260812 14:19:27.241988 21632 ts_manager.cc:194] Registered new tserver with Master: 6a44cfb795334c80955ceedba22cf754 (127.8.85.2:42385)
I20260812 14:19:27.242480 21632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:60081
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:19:27.276693 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.85.20:37087/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38199
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:33351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:19:27.407080 22030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:19:27.407466 22030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:19:27.407562 22030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:19:27.409360 22030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:19:27.409474 22030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:19:27.409559 22030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:19:27.411790 22030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:33351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.85.20:37087/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:19:27.412043 22030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:19:27.412366 22030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:19:27.418771 22036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:19:27.419112 22039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:19:27.419333 22037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:19:27.419584 22030 server_base.cc:1034] running on GCE node
I20260812 14:19:27.419802 22030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:19:27.420073 22030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:19:27.422148 22030 hybrid_clock.cc:648] HybridClock initialized: now 1786544367422118 us; error 43 us; skew 500 ppm
Aug 12 14:19:27 dist-test-slave-5k54 krb5kdc[21084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:19:27.425521 22030 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:19:27.426028 22030 webserver.cc:460] Webserver started at http://127.8.85.3:43823/ using document root <none> and password file <none>
I20260812 14:19:27.426263 22030 fs_manager.cc:359] Metadata directory not provided
I20260812 14:19:27.426311 22030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:19:27.426431 22030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:19:27 dist-test-slave-5k54 krb5kdc[21084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:19:27.442656 22030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "4212144152ea4435882f01929a34d171"
format_stamp: "Formatted at 2026-08-12 14:19:27 on dist-test-slave-5k54"
server_key: "4ca838aae02900866a66bbc0705782ab"
server_key_iv: "a48ec37b0974931c3439ffc1422f7b80"
server_key_version: "kuduclusterkey@0"
I20260812 14:19:27.443262 22030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "4212144152ea4435882f01929a34d171"
format_stamp: "Formatted at 2026-08-12 14:19:27 on dist-test-slave-5k54"
server_key: "4ca838aae02900866a66bbc0705782ab"
server_key_iv: "a48ec37b0974931c3439ffc1422f7b80"
server_key_version: "kuduclusterkey@0"
I20260812 14:19:27.446859 22030 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260812 14:19:27.461059 22048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:19:27.461323 22030 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:19:27.461418 22030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "4212144152ea4435882f01929a34d171"
format_stamp: "Formatted at 2026-08-12 14:19:27 on dist-test-slave-5k54"
server_key: "4ca838aae02900866a66bbc0705782ab"
server_key_iv: "a48ec37b0974931c3439ffc1422f7b80"
server_key_version: "kuduclusterkey@0"
I20260812 14:19:27.461513 22030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:19:27.488487 22030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:19:27.490890 22030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:19:27.491060 22030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:19:27.491344 22030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:19:27.491770 22030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:19:27.491837 22030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:19:27.491907 22030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:19:27.491967 22030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:19:27.499847 22030 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:37387
I20260812 14:19:27.500339 22030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:19:27.502015 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 22030
I20260812 14:19:27.502121 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:19:27.502305 8532 external_mini_cluster.cc:1428] Setting key 66821280ca032aac404c91ea5a7da881
I20260812 14:19:27.503603 22172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:37387 every 8 connection(s)
Aug 12 14:19:27 dist-test-slave-5k54 krb5kdc[21084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:19:27.521306 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.501273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38199 (local address 127.8.85.3:56209)
0812 14:19:27.501467 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 14:19:27.501511 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:19:27.502269 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:19:27.502713 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:19:27.502718 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:19:27.503005 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:19:27.503306 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:27.503314 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.504153 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.504154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:27.505009 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:27.505012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.505424 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.505425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:27.505563 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.506142 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:19:27.506152 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:19:27.507511 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:19:27.519537 (+ 12026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:19:27.519546 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:19:27.519550 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:19:27.519893 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:19:27.520145 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:19:27.520145 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:19:27.520146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:19:27.520282 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:19:27.520573 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:19:27.520579 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:19:27.520778 (+ 199us) client_negotiation.cc:769] Sending connection context
0812 14:19:27.520887 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 14:19:27.521063 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
I20260812 14:19:27.521639 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.501256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:56209 (local address 127.8.85.62:38199)
0812 14:19:27.501307 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:19:27.501309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:19:27.501616 (+ 307us) server_negotiation.cc:407] Connection header received
0812 14:19:27.502454 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:19:27.502456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:19:27.502471 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:19:27.502508 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:19:27.503384 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.504042 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.505104 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.505321 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.505407 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.507612 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:19:27.507614 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:19:27.507614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:19:27.507623 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:19:27.512807 (+ 5184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:19:27.520000 (+ 7193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:19:27.520003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:19:27.520003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:19:27.520077 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:19:27.520345 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:19:27.520346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:19:27.520346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:19:27.520485 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:19:27.520546 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:19:27.521465 (+ 919us) server_negotiation.cc:299] Negotiation successful
0812 14:19:27.521552 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:19:27.521924 22173 heartbeater.cc:347] Connected to a master server at 127.8.85.62:38199
I20260812 14:19:27.522038 22173 heartbeater.cc:464] Registering TS with master...
I20260812 14:19:27.522250 22173 heartbeater.cc:511] Master 127.8.85.62:38199 requested a full tablet report, sending...
I20260812 14:19:27.522666 21632 ts_manager.cc:194] Registered new tserver with Master: 4212144152ea4435882f01929a34d171 (127.8.85.3:37387)
I20260812 14:19:27.523208 21632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:56209
I20260812 14:19:27.528649 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:19:27.543605 21876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.531744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50026 (local address 127.8.85.62:38199)
0812 14:19:27.531794 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:19:27.531797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:19:27.535640 (+ 3843us) server_negotiation.cc:407] Connection header received
0812 14:19:27.535749 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:19:27.535750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:19:27.535762 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:19:27.535793 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:19:27.536653 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.537440 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.538635 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.538962 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.539005 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.540685 (+ 1680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:19:27.540687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:19:27.540687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:19:27.540694 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:19:27.542388 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:19:27.542745 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:19:27.542746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:19:27.542746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:19:27.542806 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:19:27.543030 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:19:27.543030 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:19:27.543031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:19:27.543147 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:19:27.543254 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 14:19:27.543387 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 14:19:27.543499 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:19:27.545488 21632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:19:27.547825 21632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:19:27.557560 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.550782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:33927 (local address 127.0.0.1:38490)
0812 14:19:27.551939 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0812 14:19:27.551949 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:19:27.551999 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:19:27.552272 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:19:27.552275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:19:27.552277 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:19:27.552338 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:27.552340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.554084 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.554085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:27.556251 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:27.556253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.557263 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.557264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:27.557346 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.557360 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 14:19:27.557435 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 14:19:27.557487 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1130,"thread_start_us":62,"threads_started":1}
I20260812 14:19:27.557883 22194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.551148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38490 (local address 127.8.85.1:33927)
0812 14:19:27.551873 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0812 14:19:27.551878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:19:27.552056 (+ 178us) server_negotiation.cc:407] Connection header received
0812 14:19:27.552095 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:19:27.552097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:19:27.552115 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:19:27.552188 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:19:27.552425 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.553988 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.556330 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.557087 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.557163 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.557225 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:19:27.557771 (+ 546us) server_negotiation.cc:299] Negotiation successful
0812 14:19:27.557811 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":54,"threads_started":1}
I20260812 14:19:27.558573 21796 tablet_service.cc:1467] Processing CreateTablet for tablet c2aef4f253be471e924716689ed5e7da (DEFAULT_TABLE table=test-table [id=287bc625de0e41208adeeb8c2f202a16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:19:27.558987 21796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2aef4f253be471e924716689ed5e7da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:19:27.570492 22192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.550537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:37387 (local address 127.0.0.1:56648)
0812 14:19:27.551626 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0812 14:19:27.551667 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:19:27.551731 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:19:27.556057 (+ 4326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:19:27.556058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:19:27.556060 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:19:27.556099 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:27.556100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.564748 (+ 8648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.564749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:27.565902 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:27.565905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.570252 (+ 4347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.570253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:27.570322 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.570327 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:19:27.570361 (+ 34us) client_negotiation.cc:240] Negotiation successful
0812 14:19:27.570403 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1058,"thread_start_us":99,"threads_started":1}
I20260812 14:19:27.570969 22189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.550646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56648 (local address 127.8.85.3:37387)
0812 14:19:27.555820 (+ 5174us) server_negotiation.cc:206] Beginning negotiation
0812 14:19:27.555825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:19:27.555835 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:19:27.555872 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:19:27.555874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:19:27.555890 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:19:27.555975 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:19:27.563342 (+ 7367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.564470 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.566519 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.570144 (+ 3625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.570705 (+ 561us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.570805 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 14:19:27.570831 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 14:19:27.570876 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5134,"thread_start_us":71,"threads_started":1}
I20260812 14:19:27.571712 22101 tablet_service.cc:1467] Processing CreateTablet for tablet c2aef4f253be471e924716689ed5e7da (DEFAULT_TABLE table=test-table [id=287bc625de0e41208adeeb8c2f202a16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:19:27.572144 22101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2aef4f253be471e924716689ed5e7da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:19:27.576354 22195 tablet_bootstrap.cc:492] T c2aef4f253be471e924716689ed5e7da P 46e0fa74015c4973a9b6cd9a737d37fa: Bootstrap starting.
I20260812 14:19:27.577280 22195 tablet_bootstrap.cc:654] T c2aef4f253be471e924716689ed5e7da P 46e0fa74015c4973a9b6cd9a737d37fa: Neither blocks nor log segments found. Creating new log.
I20260812 14:19:27.577665 22195 log.cc:824] T c2aef4f253be471e924716689ed5e7da P 46e0fa74015c4973a9b6cd9a737d37fa: Log is configured to *not* fsync() on all Append() calls
I20260812 14:19:27.579314 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.550957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:42385 (local address 127.0.0.1:34000)
0812 14:19:27.564613 (+ 13656us) negotiation.cc:107] Waiting for socket to connect
0812 14:19:27.564623 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:19:27.564661 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:19:27.566359 (+ 1698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:19:27.566360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:19:27.566362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:19:27.566394 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:27.566395 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.568676 (+ 2281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.568677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:27.569756 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:27.569759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.579052 (+ 9293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.579053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:27.579127 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.579131 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:19:27.579188 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 14:19:27.579239 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13642,"thread_start_us":65,"threads_started":1}
I20260812 14:19:27.579670 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.551069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34000 (local address 127.8.85.2:42385)
0812 14:19:27.566105 (+ 15036us) server_negotiation.cc:206] Beginning negotiation
0812 14:19:27.566109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:19:27.566120 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:19:27.566160 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:19:27.566162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:19:27.566178 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:19:27.566262 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:19:27.567502 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.568575 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.578006 (+ 9431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.578830 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.578914 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.579010 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 14:19:27.579541 (+ 531us) server_negotiation.cc:299] Negotiation successful
0812 14:19:27.579586 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14988,"thread_start_us":73,"threads_started":1}
I20260812 14:19:27.580339 21952 tablet_service.cc:1467] Processing CreateTablet for tablet c2aef4f253be471e924716689ed5e7da (DEFAULT_TABLE table=test-table [id=287bc625de0e41208adeeb8c2f202a16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:19:27.580715 21952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2aef4f253be471e924716689ed5e7da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:19:27.584925 22197 tablet_bootstrap.cc:492] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171: Bootstrap starting.
I20260812 14:19:27.586105 22197 tablet_bootstrap.cc:654] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171: Neither blocks nor log segments found. Creating new log.
I20260812 14:19:27.586439 22197 log.cc:824] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171: Log is configured to *not* fsync() on all Append() calls
I20260812 14:19:27.587311 22195 tablet_bootstrap.cc:492] T c2aef4f253be471e924716689ed5e7da P 46e0fa74015c4973a9b6cd9a737d37fa: No bootstrap required, opened a new log
I20260812 14:19:27.587404 22195 ts_tablet_manager.cc:1397] T c2aef4f253be471e924716689ed5e7da P 46e0fa74015c4973a9b6cd9a737d37fa: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260812 14:19:27.590173 22195 raft_consensus.cc:348] T c2aef4f253be471e924716689ed5e7da P 46e0fa74015c4973a9b6cd9a737d37fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4212144152ea4435882f01929a34d171" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37387 } } peers { permanent_uuid: "46e0fa74015c4973a9b6cd9a737d37fa" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33927 } } peers { permanent_uuid: "6a44cfb795334c80955ceedba22cf754" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42385 } }
I20260812 14:19:27.590368 22195 raft_consensus.cc:374] T c2aef4f253be471e924716689ed5e7da P 46e0fa74015c4973a9b6cd9a737d37fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:19:27.590417 22195 raft_consensus.cc:729] T c2aef4f253be471e924716689ed5e7da P 46e0fa74015c4973a9b6cd9a737d37fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46e0fa74015c4973a9b6cd9a737d37fa, State: Initialized, Role: FOLLOWER
I20260812 14:19:27.590667 22200 tablet_bootstrap.cc:492] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754: Bootstrap starting.
I20260812 14:19:27.591758 22195 consensus_queue.cc:260] T c2aef4f253be471e924716689ed5e7da P 46e0fa74015c4973a9b6cd9a737d37fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4212144152ea4435882f01929a34d171" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37387 } } peers { permanent_uuid: "46e0fa74015c4973a9b6cd9a737d37fa" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33927 } } peers { permanent_uuid: "6a44cfb795334c80955ceedba22cf754" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42385 } }
I20260812 14:19:27.592098 22195 ts_tablet_manager.cc:1428] T c2aef4f253be471e924716689ed5e7da P 46e0fa74015c4973a9b6cd9a737d37fa: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260812 14:19:27.592223 21872 heartbeater.cc:503] Master 127.8.85.62:38199 was elected leader, sending a full tablet report...
I20260812 14:19:27.593778 22200 tablet_bootstrap.cc:654] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754: Neither blocks nor log segments found. Creating new log.
I20260812 14:19:27.594156 22200 log.cc:824] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754: Log is configured to *not* fsync() on all Append() calls
I20260812 14:19:27.595101 22197 tablet_bootstrap.cc:492] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171: No bootstrap required, opened a new log
I20260812 14:19:27.595191 22197 ts_tablet_manager.cc:1397] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260812 14:19:27.597455 22197 raft_consensus.cc:348] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4212144152ea4435882f01929a34d171" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37387 } } peers { permanent_uuid: "46e0fa74015c4973a9b6cd9a737d37fa" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33927 } } peers { permanent_uuid: "6a44cfb795334c80955ceedba22cf754" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42385 } }
I20260812 14:19:27.597651 22197 raft_consensus.cc:374] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:19:27.597699 22197 raft_consensus.cc:729] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4212144152ea4435882f01929a34d171, State: Initialized, Role: FOLLOWER
I20260812 14:19:27.597863 22197 consensus_queue.cc:260] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4212144152ea4435882f01929a34d171" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37387 } } peers { permanent_uuid: "46e0fa74015c4973a9b6cd9a737d37fa" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33927 } } peers { permanent_uuid: "6a44cfb795334c80955ceedba22cf754" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42385 } }
I20260812 14:19:27.598188 22197 ts_tablet_manager.cc:1428] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:19:27.598320 22173 heartbeater.cc:503] Master 127.8.85.62:38199 was elected leader, sending a full tablet report...
I20260812 14:19:27.600328 22200 tablet_bootstrap.cc:492] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754: No bootstrap required, opened a new log
I20260812 14:19:27.600421 22200 ts_tablet_manager.cc:1397] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260812 14:19:27.603112 22200 raft_consensus.cc:348] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4212144152ea4435882f01929a34d171" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37387 } } peers { permanent_uuid: "46e0fa74015c4973a9b6cd9a737d37fa" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33927 } } peers { permanent_uuid: "6a44cfb795334c80955ceedba22cf754" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42385 } }
I20260812 14:19:27.603303 22200 raft_consensus.cc:374] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:19:27.603350 22200 raft_consensus.cc:729] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a44cfb795334c80955ceedba22cf754, State: Initialized, Role: FOLLOWER
I20260812 14:19:27.603523 22200 consensus_queue.cc:260] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4212144152ea4435882f01929a34d171" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37387 } } peers { permanent_uuid: "46e0fa74015c4973a9b6cd9a737d37fa" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33927 } } peers { permanent_uuid: "6a44cfb795334c80955ceedba22cf754" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42385 } }
I20260812 14:19:27.603837 22200 ts_tablet_manager.cc:1428] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:19:27.603972 22024 heartbeater.cc:503] Master 127.8.85.62:38199 was elected leader, sending a full tablet report...
I20260812 14:19:27.672354 22205 raft_consensus.cc:482] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:19:27.672500 22205 raft_consensus.cc:504] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4212144152ea4435882f01929a34d171" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37387 } } peers { permanent_uuid: "46e0fa74015c4973a9b6cd9a737d37fa" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33927 } } peers { permanent_uuid: "6a44cfb795334c80955ceedba22cf754" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42385 } }
I20260812 14:19:27.672925 22205 leader_election.cc:288] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46e0fa74015c4973a9b6cd9a737d37fa (127.8.85.1:33927), 6a44cfb795334c80955ceedba22cf754 (127.8.85.2:42385)
I20260812 14:19:27.677199 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.673132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:33927 (local address 127.8.85.3:34331)
0812 14:19:27.673187 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 14:19:27.673201 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:19:27.673257 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:19:27.673791 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:19:27.673792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:19:27.673794 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:19:27.673889 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:27.673890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.674825 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.674826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:27.676110 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:27.676113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.676946 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.676947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:27.677038 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.677041 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:19:27.677089 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 14:19:27.677125 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 14:19:27.677722 22194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.673504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:34331 (local address 127.8.85.1:33927)
0812 14:19:27.673597 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 14:19:27.673599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:19:27.673608 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:19:27.673637 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:19:27.673638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:19:27.673648 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:19:27.673703 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:19:27.673973 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.674733 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.676209 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.676836 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.676903 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.676914 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:19:27.677614 (+ 700us) server_negotiation.cc:299] Negotiation successful
0812 14:19:27.677659 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Aug 12 14:19:27 dist-test-slave-5k54 krb5kdc[21084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544367, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:19:27.678678 21816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2aef4f253be471e924716689ed5e7da" candidate_uuid: "4212144152ea4435882f01929a34d171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46e0fa74015c4973a9b6cd9a737d37fa" is_pre_election: true
I20260812 14:19:27.678845 21816 raft_consensus.cc:2454] T c2aef4f253be471e924716689ed5e7da P 46e0fa74015c4973a9b6cd9a737d37fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4212144152ea4435882f01929a34d171 in term 0.
I20260812 14:19:27.679122 22053 leader_election.cc:302] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4212144152ea4435882f01929a34d171, 46e0fa74015c4973a9b6cd9a737d37fa; no voters:
I20260812 14:19:27.679265 22205 raft_consensus.cc:2781] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:19:27.679322 22205 raft_consensus.cc:482] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:19:27.679353 22205 raft_consensus.cc:3037] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:19:27.680348 22205 raft_consensus.cc:504] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4212144152ea4435882f01929a34d171" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37387 } } peers { permanent_uuid: "46e0fa74015c4973a9b6cd9a737d37fa" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33927 } } peers { permanent_uuid: "6a44cfb795334c80955ceedba22cf754" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42385 } }
I20260812 14:19:27.680505 22205 leader_election.cc:288] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [CANDIDATE]: Term 1 election: Requested vote from peers 46e0fa74015c4973a9b6cd9a737d37fa (127.8.85.1:33927), 6a44cfb795334c80955ceedba22cf754 (127.8.85.2:42385)
I20260812 14:19:27.680707 21816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2aef4f253be471e924716689ed5e7da" candidate_uuid: "4212144152ea4435882f01929a34d171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46e0fa74015c4973a9b6cd9a737d37fa"
I20260812 14:19:27.680789 21816 raft_consensus.cc:3037] T c2aef4f253be471e924716689ed5e7da P 46e0fa74015c4973a9b6cd9a737d37fa [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:19:27.681702 21816 raft_consensus.cc:2454] T c2aef4f253be471e924716689ed5e7da P 46e0fa74015c4973a9b6cd9a737d37fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4212144152ea4435882f01929a34d171 in term 1.
I20260812 14:19:27.681965 22053 leader_election.cc:302] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4212144152ea4435882f01929a34d171, 46e0fa74015c4973a9b6cd9a737d37fa; no voters:
I20260812 14:19:27.682065 22205 raft_consensus.cc:2781] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:19:27.682220 22205 raft_consensus.cc:686] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [term 1 LEADER]: Becoming Leader. State: Replica: 4212144152ea4435882f01929a34d171, State: Running, Role: LEADER
I20260812 14:19:27.682319 22205 consensus_queue.cc:237] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4212144152ea4435882f01929a34d171" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37387 } } peers { permanent_uuid: "46e0fa74015c4973a9b6cd9a737d37fa" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33927 } } peers { permanent_uuid: "6a44cfb795334c80955ceedba22cf754" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42385 } }
I20260812 14:19:27.683225 21632 catalog_manager.cc:5495] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4212144152ea4435882f01929a34d171 (127.8.85.3). New cstate: current_term: 1 leader_uuid: "4212144152ea4435882f01929a34d171" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4212144152ea4435882f01929a34d171" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46e0fa74015c4973a9b6cd9a737d37fa" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a44cfb795334c80955ceedba22cf754" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42385 } health_report { overall_health: UNKNOWN } } }
I20260812 14:19:27.687567 22193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.673554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:52263 (local address 127.8.85.2:42385)
0812 14:19:27.673758 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 14:19:27.673760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:19:27.677404 (+ 3644us) server_negotiation.cc:407] Connection header received
0812 14:19:27.677437 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:19:27.677438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:19:27.677448 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:19:27.677502 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:19:27.684194 (+ 6692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.684966 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.686582 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.687125 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.687191 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.687202 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:19:27.687466 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 14:19:27.687504 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185}
I20260812 14:19:27.687740 22175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.673228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:42385 (local address 127.8.85.3:52263)
0812 14:19:27.677282 (+ 4054us) negotiation.cc:107] Waiting for socket to connect
0812 14:19:27.677294 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:19:27.677351 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:19:27.683775 (+ 6424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:19:27.683777 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:19:27.683779 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:19:27.683817 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:27.683819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.685357 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.685358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:27.686462 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:19:27.686465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.687235 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.687236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:19:27.687297 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.687300 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:19:27.687357 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 14:19:27.687392 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4043,"thread_start_us":116,"threads_started":1}
I20260812 14:19:27.688095 21974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2aef4f253be471e924716689ed5e7da" candidate_uuid: "4212144152ea4435882f01929a34d171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a44cfb795334c80955ceedba22cf754" is_pre_election: true
I20260812 14:19:27.688249 21974 raft_consensus.cc:2454] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4212144152ea4435882f01929a34d171 in term 0.
I20260812 14:19:27.688375 21973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2aef4f253be471e924716689ed5e7da" candidate_uuid: "4212144152ea4435882f01929a34d171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a44cfb795334c80955ceedba22cf754"
I20260812 14:19:27.688442 21973 raft_consensus.cc:3037] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:19:27.689124 21973 raft_consensus.cc:2454] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4212144152ea4435882f01929a34d171 in term 1.
I20260812 14:19:27.720319 22189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.715885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.8.85.3:37387)
0812 14:19:27.715937 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:19:27.715939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:19:27.716007 (+ 68us) server_negotiation.cc:407] Connection header received
0812 14:19:27.716129 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:19:27.716130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:19:27.716155 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:19:27.716225 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:19:27.716461 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.717257 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.718725 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:19:27.719045 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:19:27.719108 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:19:27.719692 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:19:27.719839 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:19:27.719924 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:19:27.720174 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 14:19:27.720199 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:19:27.723387 21973 raft_consensus.cc:1261] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754 [term 1 FOLLOWER]: Refusing update from remote peer 4212144152ea4435882f01929a34d171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:19:27.723814 22208 consensus_queue.cc:1035] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a44cfb795334c80955ceedba22cf754" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:19:27.726310 21816 raft_consensus.cc:1261] T c2aef4f253be471e924716689ed5e7da P 46e0fa74015c4973a9b6cd9a737d37fa [term 1 FOLLOWER]: Refusing update from remote peer 4212144152ea4435882f01929a34d171: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:19:27.726779 22208 consensus_queue.cc:1035] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46e0fa74015c4973a9b6cd9a737d37fa" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 14:19:27.727217 21873 tablet.cc:2369] T c2aef4f253be471e924716689ed5e7da P 46e0fa74015c4973a9b6cd9a737d37fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:19:27.728534 22025 tablet.cc:2369] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:19:27.739588 22212 mvcc.cc:205] Tried to move back new op lower bound from 7317685730189389824 to 7317685730027474944. Current Snapshot: MvccSnapshot[applied={T|T < 7317685730189389824}]
I20260812 14:19:27.739717 22213 mvcc.cc:205] Tried to move back new op lower bound from 7317685730189389824 to 7317685730027474944. Current Snapshot: MvccSnapshot[applied={T|T < 7317685730189389824}]
I20260812 14:19:27.741341 22216 mvcc.cc:205] Tried to move back new op lower bound from 7317685730189389824 to 7317685730027474944. Current Snapshot: MvccSnapshot[applied={T|T < 7317685730189389824}]
I20260812 14:19:27.744616 21632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:19:27.744720 21632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:19:27.745484 21632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6416ca4b6cc44b5e879b50688aa0ca19: Sending DeleteTablet for 3 replicas of tablet c2aef4f253be471e924716689ed5e7da
I20260812 14:19:27.746009 22101 tablet_service.cc:1514] Processing DeleteTablet for tablet c2aef4f253be471e924716689ed5e7da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:19:27 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:56648
I20260812 14:19:27.746239 21796 tablet_service.cc:1514] Processing DeleteTablet for tablet c2aef4f253be471e924716689ed5e7da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:19:27 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:38490
I20260812 14:19:27.746436 21952 tablet_service.cc:1514] Processing DeleteTablet for tablet c2aef4f253be471e924716689ed5e7da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:19:27 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:34000
I20260812 14:19:27.747169 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 21726
I20260812 14:19:27.753912 22227 tablet_replica.cc:330] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754: stopping tablet replica
I20260812 14:19:27.754057 22224 tablet_replica.cc:330] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171: stopping tablet replica
I20260812 14:19:27.754426 22227 raft_consensus.cc:2229] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:19:27.754317 22224 raft_consensus.cc:2229] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:19:27.754709 22224 raft_consensus.cc:2258] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:19:27.754868 22227 raft_consensus.cc:2258] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:19:27.755589 22227 ts_tablet_manager.cc:1905] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:19:27.756533 22224 ts_tablet_manager.cc:1905] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:19:27.759712 22227 ts_tablet_manager.cc:1918] T c2aef4f253be471e924716689ed5e7da P 6a44cfb795334c80955ceedba22cf754: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260812 14:19:27.763831 21619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:33927 (error 108)
I20260812 14:19:27.764753 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.764638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:33927 (local address 127.0.0.1:38492)
0812 14:19:27.764688 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 14:19:27.764719 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:33927: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 14:19:27.765002 21619 catalog_manager.cc:4584] TS 46e0fa74015c4973a9b6cd9a737d37fa (127.8.85.1:33927): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c2aef4f253be471e924716689ed5e7da: Network error: Client connection negotiation failed: client connection to 127.8.85.1:33927: connect: Connection refused (error 111)
I20260812 14:19:27.765241 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 21880
I20260812 14:19:27.767563 22224 ts_tablet_manager.cc:1918] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:19:27.767645 22224 log.cc:1196] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/c2aef4f253be471e924716689ed5e7da
I20260812 14:19:27.767951 22224 ts_tablet_manager.cc:1939] T c2aef4f253be471e924716689ed5e7da P 4212144152ea4435882f01929a34d171: Deleting consensus metadata
I20260812 14:19:27.773918 21619 catalog_manager.cc:4841] TS 4212144152ea4435882f01929a34d171 (127.8.85.3:37387): tablet c2aef4f253be471e924716689ed5e7da (table test-table [id=287bc625de0e41208adeeb8c2f202a16]) successfully deleted
I20260812 14:19:27.786260 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 22030
I20260812 14:19:27.790179 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:19:27.786376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:42385 (local address 127.0.0.1:34010)
0812 14:19:27.787401 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0812 14:19:27.790130 (+ 2729us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.2:42385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":999}
I20260812 14:19:27.805063 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 21593
I20260812 14:19:27.814908 8532 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 14:19:28.159144 8532 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 14:19:28.159253 8532 mini_ranger.cc:66] Stopping Ranger...
I20260812 14:19:28.503558 8532 mini_ranger.cc:68] Stopped Ranger
2026-08-12 14:19:28.503 UTC [21121] LOG: received smart shutdown request
2026-08-12 14:19:28.507 UTC [21121] LOG: background worker "logical replication launcher" (PID 21130) exited with exit code 1
2026-08-12 14:19:28.508 UTC [21125] LOG: shutting down
2026-08-12 14:19:28.668 UTC [21121] LOG: database system is shut down
2026-08-12T14:19:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46665 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:19:28 dist-test-slave-5k54 krb5kdc[22237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:19:28 dist-test-slave-5k54 krb5kdc[22237](info): set up 2 sockets
Aug 12 14:19:28 dist-test-slave-5k54 krb5kdc[22237](info): commencing operation
krb5kdc: starting...
W20260812 14:19:29.901089 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:19:29 dist-test-slave-5k54 krb5kdc[22237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:19:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:19:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.85.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.85.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/rangeradmin_127.8.85.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/rangeradmin_127.8.85.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.85.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.85.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/rangerlookup_127.8.85.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/rangerlookup_127.8.85.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.85.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.85.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/HTTP_127.8.85.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/HTTP_127.8.85.20@KRBTEST.COM.keytab.
I20260812 14:19:30.069802 8532 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskYVQ8wa/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/postgres -l logfile start
2026-08-12 14:19:32.479 UTC [22269] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 14:19:32.480 UTC [22269] LOG: listening on IPv4 address "127.8.85.20", port 46935
2026-08-12 14:19:32.483 UTC [22269] LOG: listening on Unix socket "/tmp/.s.PGSQL.46935"
2026-08-12 14:19:32.505 UTC [22273] LOG: database system was shut down at 2026-08-12 14:19:31 UTC
2026-08-12 14:19:32.509 UTC [22269] LOG: database system is ready to accept connections
I20260812 14:19:33.536854 8532 mini_postgres.cc:96] Postgres bound to 46935
2026-08-12 14:19:33.544 UTC [22282] FATAL: database "slave" does not exist
127.8.85.20:46935 - accepting connections
I20260812 14:19:33.544950 8532 mini_ranger.cc:161] Starting Ranger...
I20260812 14:19:33.560289 8532 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 14:19:34.049253 8532 mini_ranger.cc:87] Created ranger Postgres database
I20260812 14:19:34.049340 8532 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-admin
2026-08-12 14:19:34,339 [I] DB FLAVOR :POSTGRES
2026-08-12 14:19:34,340 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 14:19:34,340 [I] Checking connection..
2026-08-12 14:19:34,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 14:19:34,609 [I] Checking connection passed.
2026-08-12 14:19:34,609 [I] --------- Verifying version history table ---------
2026-08-12 14:19:34,609 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:19:34,863 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 14:19:34,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:19:35,098 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 14:19:35,098 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 14:19:35,098 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 14:19:35.323 UTC [22354] WARNING: there is no transaction in progress
2026-08-12 14:19:35,331 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 14:19:35,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:19:35,587 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 14:19:35,587 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 14:19:35,587 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 14:19:35,848 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 14:19:36,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5k54.c.gcp-upstream.internal','N') ;"
2026-08-12 14:19:36,368 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 14:19:36,614 [I] Table x_portal_user does not exist in database ranger
2026-08-12 14:19:36,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 14:19:36,882 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 14:19:36,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 14:19:37,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 14:19:37,388 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 14:19:37,389 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 14:19:38.364 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:38.381 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:38.400 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:38.418 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:38.436 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:38.455 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:38.521 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:38.529 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:38.543 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:38.554 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:38.565 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:38.578 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:38.588 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:38.594 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:38.601 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:38.608 UTC [22472] WARNING: there is no transaction in progress
2026-08-12 14:19:39,294 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 14:19:39,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 14:19:39,552 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 14:19:39,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 14:19:39,816 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 14:19:39,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 14:19:40,077 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 14:19:40,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 14:19:40,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 14:19:40,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5k54.c.gcp-upstream.internal';"
2026-08-12 14:19:40,807 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 14:19:40,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 14:19:41,062 [I] DB_PATCHES have already been applied
I20260812 14:19:41.068843 8532 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-admin:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskYVQ8wa/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 14:19:41.068948 8532 mini_ranger.cc:193] Using host: 127.8.85.20
I20260812 14:19:41.073105 8532 mini_ranger.cc:227] Ranger admin URL: http://127.8.85.20:40675
Aug 12, 2026 2:19:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:19:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:19:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 2:19:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 2:19:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 2:19:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 2:19:41 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 14:19:42.267076 8532 mini_ranger.cc:160] Time spent starting Ranger: real 8.722s user 0.002s sys 0.009s
Aug 12, 2026 2:20:01 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 2:20:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 2:20:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 2:20:01 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 2:20:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 2:20:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 2:20:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 2:20:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 2:20:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 2:20:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 2:20:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 2:20:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 2:20:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 2:20:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 2:20:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 2:20:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 2:20:01 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 2:20:02 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 14:20:03.386065 8532 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.85.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.85.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/rangerkms_127.8.85.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.85.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/rangerkms_127.8.85.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.85.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.85.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.85.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/HTTP_127.8.85.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.85.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/HTTP_127.8.85.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 14:20:03 dist-test-slave-5k54 krb5kdc[22237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544403, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 14:20:03.475924 8532 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 14:20:03.486620 8532 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 14:20:04.377532 8532 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 14:20:04.377620 8532 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-kms
I20260812 14:20:04.377640 8532 mini_ranger_kms.cc:227] Using postgres at 127.8.85.20:46935
2026-08-12 14:20:04,559 [I] DB FLAVOR :POSTGRES
2026-08-12 14:20:04,560 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 14:20:04,560 [I] Checking connection
2026-08-12 14:20:04,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 14:20:04,816 [I] connection success
2026-08-12 14:20:04,816 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 14:20:04,816 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 14:20:04,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 14:20:05,070 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 14:20:05,070 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 14:20:05,070 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 14:20:05,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.85.20:46935/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 14:20:05,334 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 14:20:05.787145 8532 mini_ranger_kms.cc:310] Created kms service
I20260812 14:20:05.988196 8532 mini_ranger_kms.cc:326] Created kudu user
I20260812 14:20:06.032519 8532 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 14:20:06 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544403, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:20:06.428785 8532 mini_ranger_kms.cc:384] Added ranger policy
I20260812 14:20:06.428910 8532 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-kms:/tmp/dist-test-taskYVQ8wa/build/release/bin/postgresql.jar:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskYVQ8wa/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 14:20:06.428928 8532 mini_ranger_kms.cc:240] Using host: 127.8.85.20
I20260812 14:20:06.433790 8532 mini_ranger_kms.cc:276] Ranger KMS PID: 22676
I20260812 14:20:06.433881 8532 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.85.20:52635
Aug 12, 2026 2:20:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:20:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:20:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 2:20:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 2:20:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 2:20:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 14:20:08.865531 8532 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.390s user 0.005s sys 0.156s
I20260812 14:20:08.865638 8532 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 14:20:08.865689 8532 mini_ranger_kms.cc:401] 127.8.85.20:52635/kms/v1/keys
Aug 12 14:20:08 dist-test-slave-5k54 krb5kdc[22237](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786544408, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.85.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 14:20:09 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544408, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.85.20@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
Aug 12 14:20:09 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544408, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.85.20@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
Aug 12 14:20:12 dist-test-slave-5k54 krb5kdc[22237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:20:12.109817 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:40297
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:40297
--ranger_config_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.85.20:52635/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:12.217721 22736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:12.217935 22736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:12.217957 22736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:12.219108 22736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:20:12.219152 22736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:12.219166 22736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:12.219179 22736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:20:12.219228 22736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:20:12.220841 22736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.85.20:52635/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:40297
--ranger_config_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:40297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:12.221103 22736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:12.221354 22736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:12.226433 22741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:12.226548 22736 server_base.cc:1034] running on GCE node
W20260812 14:20:12.226449 22742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:12.226428 22744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:12.227006 22736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:12.227340 22736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:12.228510 22736 hybrid_clock.cc:648] HybridClock initialized: now 1786544412228495 us; error 30 us; skew 500 ppm
Aug 12 14:20:12 dist-test-slave-5k54 krb5kdc[22237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544412, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:12.236773 22736 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:20:12.237344 22736 webserver.cc:460] Webserver started at http://127.8.85.62:46103/ using document root <none> and password file <none>
I20260812 14:20:12.237578 22736 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:12.237639 22736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:12.237783 22736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:20:12 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544412, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:20:12.283659 22736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "6590d5218e78421cbe3a81d0bd3e3a0c"
format_stamp: "Formatted at 2026-08-12 14:20:12 on dist-test-slave-5k54"
server_key: "e4d6d3c05c37b1f0452a0af027ac1146"
server_key_iv: "4e2fa8890f77c097868e67c6baea5852"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:12.284101 22736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "6590d5218e78421cbe3a81d0bd3e3a0c"
format_stamp: "Formatted at 2026-08-12 14:20:12 on dist-test-slave-5k54"
server_key: "e4d6d3c05c37b1f0452a0af027ac1146"
server_key_iv: "4e2fa8890f77c097868e67c6baea5852"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:12.285827 22736 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 14:20:12.297775 22754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:12.298017 22736 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:12.298130 22736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "6590d5218e78421cbe3a81d0bd3e3a0c"
format_stamp: "Formatted at 2026-08-12 14:20:12 on dist-test-slave-5k54"
server_key: "e4d6d3c05c37b1f0452a0af027ac1146"
server_key_iv: "4e2fa8890f77c097868e67c6baea5852"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:12.298249 22736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:12.315215 22736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:12.317384 22736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:12.317552 22736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:12.322541 22736 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:40297
I20260812 14:20:12.322623 22815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:40297 every 8 connection(s)
I20260812 14:20:12.322981 22736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:20:12.323777 22816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:12.326116 22816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c: Bootstrap starting.
I20260812 14:20:12.326732 22816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:12.327041 22816 log.cc:824] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:12.327230 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 22736
I20260812 14:20:12.327322 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:20:12.327550 8532 external_mini_cluster.cc:1428] Setting key cefcf9ea761d9bda6f0020da0d863b6c
I20260812 14:20:12.327876 22816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c: No bootstrap required, opened a new log
I20260812 14:20:12.330181 22816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40297 } }
I20260812 14:20:12.330349 22816 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:12.330417 22816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6590d5218e78421cbe3a81d0bd3e3a0c, State: Initialized, Role: FOLLOWER
I20260812 14:20:12.330590 22816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40297 } }
I20260812 14:20:12.330693 22816 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:20:12.330729 22816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:20:12.330808 22816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:12.331626 22816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40297 } }
I20260812 14:20:12.331753 22816 leader_election.cc:302] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6590d5218e78421cbe3a81d0bd3e3a0c; no voters:
I20260812 14:20:12.331933 22816 leader_election.cc:288] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:20:12.331985 22821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:20:12.332182 22821 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 1 LEADER]: Becoming Leader. State: Replica: 6590d5218e78421cbe3a81d0bd3e3a0c, State: Running, Role: LEADER
I20260812 14:20:12.332289 22816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:20:12.332309 22821 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40297 } }
I20260812 14:20:12.332626 22822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6590d5218e78421cbe3a81d0bd3e3a0c. Latest consensus state: current_term: 1 leader_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40297 } } }
I20260812 14:20:12.332840 22822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [sys.catalog]: This master's current role is: LEADER
I20260812 14:20:12.332765 22823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40297 } } }
I20260812 14:20:12.334214 22823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [sys.catalog]: This master's current role is: LEADER
I20260812 14:20:12.334364 22836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:20:12.334156 22816 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 12 14:20:12 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:12.340003 22836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:20:12.344403 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:12.328078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51226 (local address 127.8.85.62:40297)
0812 14:20:12.328253 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:12.328263 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:12.328298 (+ 35us) server_negotiation.cc:407] Connection header received
0812 14:20:12.330183 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:12.330193 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:12.330272 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:12.330424 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:12.331346 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:12.332869 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:12.334031 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:12.334669 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:12.334778 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:12.338507 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:12.338549 (+ 42us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:12.338549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:12.338650 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:12.341731 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:12.342325 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:12.342329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:12.342329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:12.342455 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:12.342661 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:12.342662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:12.342663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:12.343094 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:12.343254 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:12.343577 (+ 323us) server_negotiation.cc:299] Negotiation successful
0812 14:20:12.343900 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260812 14:20:12.351955 22836 catalog_manager.cc:1347] Generated new cluster ID: 52be9ef506ba427f886d21eabe2fb70b
I20260812 14:20:12.352011 22836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:20:12.362249 22836 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:20:12.363107 22836 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:20:12.368180 22836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c: Generated new TSK 0
I20260812 14:20:12.368427 22836 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:20:13.156848 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.85.20:52635/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:40297
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:13.286435 22865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:13.286803 22865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:13.286881 22865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:13.288673 22865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:13.288789 22865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:13.288870 22865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:20:13.291054 22865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.85.20:52635/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:40297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:13.291584 22865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:13.291949 22865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:13.298424 22872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:13.298471 22871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:13.298667 22875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:13.299501 22865 server_base.cc:1034] running on GCE node
I20260812 14:20:13.299728 22865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:13.300011 22865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:13.301185 22865 hybrid_clock.cc:648] HybridClock initialized: now 1786544413301164 us; error 32 us; skew 500 ppm
Aug 12 14:20:13 dist-test-slave-5k54 krb5kdc[22237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:13.304644 22865 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:20:13.305155 22865 webserver.cc:460] Webserver started at http://127.8.85.1:41863/ using document root <none> and password file <none>
I20260812 14:20:13.305478 22865 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:13.305542 22865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:13.305702 22865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:20:13 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:20:13.320412 22865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "32390f9513f2431b8f373b205da83ebe"
format_stamp: "Formatted at 2026-08-12 14:20:13 on dist-test-slave-5k54"
server_key: "98462fe2969e07e15a4993321f589198"
server_key_iv: "26e32ba388b82adb3df222318c4c14ec"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:13.320828 22865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "32390f9513f2431b8f373b205da83ebe"
format_stamp: "Formatted at 2026-08-12 14:20:13 on dist-test-slave-5k54"
server_key: "98462fe2969e07e15a4993321f589198"
server_key_iv: "26e32ba388b82adb3df222318c4c14ec"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:13.323803 22865 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 14:20:13.338268 22886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:13.338526 22865 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:13.338641 22865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "32390f9513f2431b8f373b205da83ebe"
format_stamp: "Formatted at 2026-08-12 14:20:13 on dist-test-slave-5k54"
server_key: "98462fe2969e07e15a4993321f589198"
server_key_iv: "26e32ba388b82adb3df222318c4c14ec"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:13.338735 22865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:13.384902 22865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:13.387400 22865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:13.387643 22865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:13.387996 22865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:13.388468 22865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:13.388532 22865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:13.388603 22865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:13.388649 22865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:13.396927 22865 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:42119
I20260812 14:20:13.397329 22865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:20:13.397985 23015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:42119 every 8 connection(s)
I20260812 14:20:13.403177 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 22865
I20260812 14:20:13.403321 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:20:13.403709 8532 external_mini_cluster.cc:1428] Setting key b26c05c8bcb42dcb7063b9183572bbb2
Aug 12 14:20:13 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:13.421234 23018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:13.398124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:37683 (local address 127.8.85.62:40297)
0812 14:20:13.398940 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:13.398942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:13.398959 (+ 17us) server_negotiation.cc:407] Connection header received
0812 14:20:13.398983 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:13.398983 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:13.398994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:13.399081 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:13.399748 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:13.400478 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:13.402009 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:13.402319 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:13.403926 (+ 1607us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:13.415484 (+ 11558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:13.415486 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:13.415487 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:13.415497 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:13.417480 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:13.419956 (+ 2476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:13.419957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:13.419958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:13.420030 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:13.420349 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:13.420350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:13.420350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:13.420508 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:13.420562 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:13.421027 (+ 465us) server_negotiation.cc:299] Negotiation successful
0812 14:20:13.421123 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":59,"threads_started":1}
I20260812 14:20:13.422065 23019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:13.398095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:40297 (local address 127.8.85.1:37683)
0812 14:20:13.398260 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:13.398302 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:13.398784 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:13.399174 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:13.399179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:13.399368 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:13.399642 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:13.399648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:13.400634 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:13.400636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:13.401888 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:13.401891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:13.402430 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:13.402431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:13.402570 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:13.403208 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:13.403220 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:13.407574 (+ 4354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:13.419485 (+ 11911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:13.419495 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:13.419499 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:13.419846 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:13.420109 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:13.420110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:13.420110 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:13.420268 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:13.420597 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:13.420604 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:13.420851 (+ 247us) client_negotiation.cc:769] Sending connection context
0812 14:20:13.421636 (+ 785us) client_negotiation.cc:240] Negotiation successful
0812 14:20:13.421824 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
I20260812 14:20:13.423949 23016 heartbeater.cc:347] Connected to a master server at 127.8.85.62:40297
I20260812 14:20:13.424114 23016 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:13.424371 23016 heartbeater.cc:511] Master 127.8.85.62:40297 requested a full tablet report, sending...
I20260812 14:20:13.425004 22774 ts_manager.cc:194] Registered new tserver with Master: 32390f9513f2431b8f373b205da83ebe (127.8.85.1:42119)
I20260812 14:20:13.425920 22774 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:37683
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:20:13.467926 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.85.20:52635/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:40297
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:20:13 dist-test-slave-5k54 krb5kdc[22237](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786544413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 14:20:13.606988 23026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:13.607266 23026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:13.607301 23026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:13.609005 23026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:13.609063 23026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:13.609102 23026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:20:13.611271 23026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.85.20:52635/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:40297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:13.611555 23026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:13.611832 23026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:13.618151 23032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:13.618182 23036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:13.618232 23026 server_base.cc:1034] running on GCE node
W20260812 14:20:13.618274 23031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:13.618583 23026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:13.618866 23026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:13.620097 23026 hybrid_clock.cc:648] HybridClock initialized: now 1786544413620090 us; error 52 us; skew 500 ppm
Aug 12 14:20:13 dist-test-slave-5k54 krb5kdc[22237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:13.622859 23026 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:20:13.623443 23026 webserver.cc:460] Webserver started at http://127.8.85.2:38723/ using document root <none> and password file <none>
I20260812 14:20:13.623806 23026 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:13.623889 23026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:13.624183 23026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:20:13 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:20:13.641499 23026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "eaa7052342a44f4f8a9948f96be57278"
format_stamp: "Formatted at 2026-08-12 14:20:13 on dist-test-slave-5k54"
server_key: "c3f2067c16068940ac8ec04ba48b664a"
server_key_iv: "9b8084890c33e9bd8ef5b4dcf5a103e5"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:13.641964 23026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "eaa7052342a44f4f8a9948f96be57278"
format_stamp: "Formatted at 2026-08-12 14:20:13 on dist-test-slave-5k54"
server_key: "c3f2067c16068940ac8ec04ba48b664a"
server_key_iv: "9b8084890c33e9bd8ef5b4dcf5a103e5"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:13.644330 23026 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 14:20:13.663594 23046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:13.663889 23026 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260812 14:20:13.663993 23026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "eaa7052342a44f4f8a9948f96be57278"
format_stamp: "Formatted at 2026-08-12 14:20:13 on dist-test-slave-5k54"
server_key: "c3f2067c16068940ac8ec04ba48b664a"
server_key_iv: "9b8084890c33e9bd8ef5b4dcf5a103e5"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:13.664089 23026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:13.694772 23026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:13.697136 23026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:13.697324 23026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:13.697587 23026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:13.698015 23026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:13.698076 23026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:13.698153 23026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:13.698184 23026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:13.706539 23026 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:43679
I20260812 14:20:13.706627 23172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:43679 every 8 connection(s)
I20260812 14:20:13.707415 23026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:20:13.712276 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 23026
I20260812 14:20:13.712385 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:20:13.712587 8532 external_mini_cluster.cc:1428] Setting key e9d82c563c2ca36a86a4ea618ea14c60
Aug 12 14:20:13 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:13.719769 23018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:13.708504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:59061 (local address 127.8.85.62:40297)
0812 14:20:13.708588 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:13.708592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:13.708875 (+ 283us) server_negotiation.cc:407] Connection header received
0812 14:20:13.709609 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:13.709610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:13.709622 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:13.709669 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:13.710445 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:13.710998 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:13.712538 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:13.712880 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:13.712930 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:13.715789 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:13.715790 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:13.715791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:13.715807 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:13.717894 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:13.718408 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:13.718410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:13.718410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:13.718488 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:13.718831 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:13.718833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:13.718833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:13.719010 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:13.719095 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:13.719528 (+ 433us) server_negotiation.cc:299] Negotiation successful
0812 14:20:13.719637 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 14:20:13.719987 23175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:13.708461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:40297 (local address 127.8.85.2:59061)
0812 14:20:13.708705 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:13.708765 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:13.709499 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:13.709755 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:13.709759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:13.710039 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:13.710342 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:13.710350 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:13.711109 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:13.711110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:13.712423 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:13.712426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:13.713048 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:13.713049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:13.713202 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:13.713831 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:13.713842 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:13.715682 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:13.717991 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:13.717998 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:13.718000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:13.718308 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:13.718575 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:13.718576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:13.718576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:13.718713 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:13.719110 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:13.719118 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:13.719320 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 14:20:13.719489 (+ 169us) client_negotiation.cc:240] Negotiation successful
0812 14:20:13.719659 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":116,"threads_started":1}
I20260812 14:20:13.720664 23173 heartbeater.cc:347] Connected to a master server at 127.8.85.62:40297
I20260812 14:20:13.720780 23173 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:13.721055 23173 heartbeater.cc:511] Master 127.8.85.62:40297 requested a full tablet report, sending...
I20260812 14:20:13.721481 22774 ts_manager.cc:194] Registered new tserver with Master: eaa7052342a44f4f8a9948f96be57278 (127.8.85.2:43679)
I20260812 14:20:13.722043 22774 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:59061
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:20:13.747185 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.85.20:52635/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:40297
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:13.860154 23178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:13.860493 23178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:13.860538 23178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:13.862315 23178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:13.862380 23178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:13.862423 23178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:20:13.864454 23178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.85.20:52635/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:40297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:13.864728 23178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:13.865012 23178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:13.870154 23189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:13.870201 23187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:13.870168 23184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:13.870491 23178 server_base.cc:1034] running on GCE node
I20260812 14:20:13.870850 23178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:13.871162 23178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:13.872363 23178 hybrid_clock.cc:648] HybridClock initialized: now 1786544413872347 us; error 27 us; skew 500 ppm
Aug 12 14:20:13 dist-test-slave-5k54 krb5kdc[22237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:13.876119 23178 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:20:13.876559 23178 webserver.cc:460] Webserver started at http://127.8.85.3:45949/ using document root <none> and password file <none>
I20260812 14:20:13.876818 23178 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:13.876895 23178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:13.877048 23178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:20:13 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:20:13.900117 23178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "6b2112a43275483d869faac4d8176cf7"
format_stamp: "Formatted at 2026-08-12 14:20:13 on dist-test-slave-5k54"
server_key: "47daa2445394e6c3d3ae4a39522d8ecc"
server_key_iv: "8c6ab0a78790a09a4b943742a007e690"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:13.900719 23178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "6b2112a43275483d869faac4d8176cf7"
format_stamp: "Formatted at 2026-08-12 14:20:13 on dist-test-slave-5k54"
server_key: "47daa2445394e6c3d3ae4a39522d8ecc"
server_key_iv: "8c6ab0a78790a09a4b943742a007e690"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:13.903685 23178 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260812 14:20:13.919687 23198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:13.919950 23178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 14:20:13.920061 23178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "6b2112a43275483d869faac4d8176cf7"
format_stamp: "Formatted at 2026-08-12 14:20:13 on dist-test-slave-5k54"
server_key: "47daa2445394e6c3d3ae4a39522d8ecc"
server_key_iv: "8c6ab0a78790a09a4b943742a007e690"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:13.920156 23178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:13.933216 23178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:13.935796 23178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:13.936050 23178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:13.936369 23178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:13.936852 23178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:13.936954 23178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:13.937037 23178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:13.937103 23178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:13.945072 23178 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:40859
I20260812 14:20:13.946130 23178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:20:13.951683 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 23178
I20260812 14:20:13.951794 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:20:13.952003 8532 external_mini_cluster.cc:1428] Setting key 6df0886e79becce9f98460137807a4e6
I20260812 14:20:13.948731 23331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:40859 every 8 connection(s)
Aug 12 14:20:13 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:13.962548 23018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:13.949469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:52799 (local address 127.8.85.62:40297)
0812 14:20:13.949519 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:13.949522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:13.949790 (+ 268us) server_negotiation.cc:407] Connection header received
0812 14:20:13.950583 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:13.950585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:13.950600 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:13.950636 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:13.953156 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:13.954019 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:13.955829 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:13.956129 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:13.956179 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:13.959022 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:13.959024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:13.959024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:13.959033 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:13.960986 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:13.961470 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:13.961471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:13.961472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:13.961539 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:13.961821 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:13.961822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:13.961823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:13.961948 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:13.962030 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:13.962345 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 14:20:13.962431 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:20:13.962987 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:13.949450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:40297 (local address 127.8.85.3:52799)
0812 14:20:13.949635 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:13.949691 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:13.950434 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:13.950848 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:13.950852 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:13.951134 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:13.953044 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:13.953052 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:13.954151 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:13.954152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:13.955714 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:13.955717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:13.956316 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:13.956318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:13.956475 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:13.957113 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:13.957125 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:13.958900 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:13.961091 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:13.961097 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:13.961100 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:13.961375 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:13.961614 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:13.961615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:13.961615 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:13.961738 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:13.962021 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:13.962028 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:13.962193 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 14:20:13.962315 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 14:20:13.962490 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":101,"threads_started":1}
I20260812 14:20:13.963745 23332 heartbeater.cc:347] Connected to a master server at 127.8.85.62:40297
I20260812 14:20:13.963853 23332 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:13.964066 23332 heartbeater.cc:511] Master 127.8.85.62:40297 requested a full tablet report, sending...
I20260812 14:20:13.964504 22773 ts_manager.cc:194] Registered new tserver with Master: 6b2112a43275483d869faac4d8176cf7 (127.8.85.3:40859)
I20260812 14:20:13.964916 22773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:52799
I20260812 14:20:13.974807 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:20:13.975399 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 22865
I20260812 14:20:13.999656 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 23026
I20260812 14:20:14.007112 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 23178
I20260812 14:20:14.025800 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 22736
I20260812 14:20:14.034796 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:40297
--webserver_interface=127.8.85.62
--webserver_port=46103
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:40297
--ranger_config_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.85.20:52635/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.85.20:52635/kms with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:14.118301 23335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:14.118513 23335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:14.118536 23335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:14.119872 23335 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:20:14.119915 23335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:14.119930 23335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:14.119942 23335 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:20:14.119954 23335 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:20:14.121560 23335 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.85.20:52635/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:40297
--ranger_config_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:40297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=46103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:14.121820 23335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:14.122073 23335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:14.127106 23342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:14.127106 23344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:14.127096 23341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:14.127139 23335 server_base.cc:1034] running on GCE node
I20260812 14:20:14.127616 23335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:14.127884 23335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:14.129024 23335 hybrid_clock.cc:648] HybridClock initialized: now 1786544414129013 us; error 28 us; skew 500 ppm
Aug 12 14:20:14 dist-test-slave-5k54 krb5kdc[22237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:14.134727 23335 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:20:14.135182 23335 webserver.cc:460] Webserver started at http://127.8.85.62:46103/ using document root <none> and password file <none>
I20260812 14:20:14.135416 23335 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:14.135519 23335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 14:20:14 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:20:14.181751 23335 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:14.182628 23354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:14.182826 23335 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:14.182934 23335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "6590d5218e78421cbe3a81d0bd3e3a0c"
format_stamp: "Formatted at 2026-08-12 14:20:12 on dist-test-slave-5k54"
server_key: "e4d6d3c05c37b1f0452a0af027ac1146"
server_key_iv: "4e2fa8890f77c097868e67c6baea5852"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:14.183337 23335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:14.196931 23335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:14.198604 23335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:14.198769 23335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:14.203169 23335 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:40297
I20260812 14:20:14.203248 23409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:40297 every 8 connection(s)
I20260812 14:20:14.203657 23335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:20:14.204969 23410 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:20:14.205654 23410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c: Bootstrap starting.
I20260812 14:20:14.211103 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 23335
I20260812 14:20:14.211458 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.85.20:52635/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:42119
--local_ip_for_outbound_sockets=127.8.85.1
--tserver_master_addrs=127.8.85.62:40297
--webserver_port=41863
--webserver_interface=127.8.85.1
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.85.20:52635/kms with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:20:14.213974 23410 log.cc:824] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:14.215920 23410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:20:14.216275 23410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c: Bootstrap complete.
I20260812 14:20:14.218675 23410 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40297 } }
I20260812 14:20:14.218943 23410 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6590d5218e78421cbe3a81d0bd3e3a0c, State: Initialized, Role: FOLLOWER
I20260812 14:20:14.219139 23410 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40297 } }
I20260812 14:20:14.219205 23410 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:20:14.219264 23410 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:20:14.219321 23410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:20:14.220381 23410 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40297 } }
I20260812 14:20:14.220472 23410 leader_election.cc:302] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6590d5218e78421cbe3a81d0bd3e3a0c; no voters:
I20260812 14:20:14.220700 23410 leader_election.cc:288] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 14:20:14.220733 23414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 2 FOLLOWER]: Leader election won for term 2
I20260812 14:20:14.220855 23414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [term 2 LEADER]: Becoming Leader. State: Replica: 6590d5218e78421cbe3a81d0bd3e3a0c, State: Running, Role: LEADER
I20260812 14:20:14.220974 23410 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:20:14.221036 23414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40297 } }
I20260812 14:20:14.221396 23416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6590d5218e78421cbe3a81d0bd3e3a0c. Latest consensus state: current_term: 2 leader_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40297 } } }
I20260812 14:20:14.221474 23416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [sys.catalog]: This master's current role is: LEADER
I20260812 14:20:14.221899 23421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:20:14.222412 23421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:20:14.222543 23421 catalog_manager.cc:1259] Loaded cluster ID: 52be9ef506ba427f886d21eabe2fb70b
I20260812 14:20:14.222636 23421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:20:14.222870 23410 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 14:20:14.223945 23421 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:20:14.224139 23421 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c: Loaded TSK: 0
I20260812 14:20:14.224355 23421 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:20:14.224476 23415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6590d5218e78421cbe3a81d0bd3e3a0c" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40297 } } }
I20260812 14:20:14.224542 23415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c [sys.catalog]: This master's current role is: LEADER
W20260812 14:20:14.326856 23412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:14.327214 23412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:14.327277 23412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:14.329016 23412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:14.329092 23412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:14.329133 23412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:20:14.331508 23412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.85.20:52635/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:42119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=41863
--tserver_master_addrs=127.8.85.62:40297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:14.331897 23412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:14.332177 23412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:14.338696 23454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:14.338907 23412 server_base.cc:1034] running on GCE node
W20260812 14:20:14.338948 23452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:14.339161 23451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:14.339485 23412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:14.339779 23412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:14.344978 23412 hybrid_clock.cc:648] HybridClock initialized: now 1786544414344955 us; error 37 us; skew 500 ppm
Aug 12 14:20:14 dist-test-slave-5k54 krb5kdc[22237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:14.348459 23412 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:20:14.349071 23412 webserver.cc:460] Webserver started at http://127.8.85.1:41863/ using document root <none> and password file <none>
I20260812 14:20:14.349380 23412 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:14.349516 23412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 14:20:14 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:20:14.383919 23412 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.001s sys 0.000s
I20260812 14:20:14.388065 23463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:14.388327 23412 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260812 14:20:14.388422 23412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "32390f9513f2431b8f373b205da83ebe"
format_stamp: "Formatted at 2026-08-12 14:20:13 on dist-test-slave-5k54"
server_key: "98462fe2969e07e15a4993321f589198"
server_key_iv: "26e32ba388b82adb3df222318c4c14ec"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:14.388926 23412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:14.427620 23412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:14.429863 23412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:14.430052 23412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:14.430325 23412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:14.430743 23412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:14.430799 23412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:14.430878 23412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:14.430925 23412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:14.437721 23412 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:42119
I20260812 14:20:14.438295 23412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:20:14.439613 23599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:42119 every 8 connection(s)
I20260812 14:20:14.444787 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 23412
I20260812 14:20:14.445116 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.85.20:52635/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:43679
--local_ip_for_outbound_sockets=127.8.85.2
--tserver_master_addrs=127.8.85.62:40297
--webserver_port=38723
--webserver_interface=127.8.85.2
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.85.20:52635/kms with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:20:14 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:14.468115 23602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:14.440800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:40297 (local address 127.8.85.1:45255)
0812 14:20:14.443670 (+ 2870us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:14.443707 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:14.444249 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:14.444972 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:14.444977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:14.445239 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:14.445534 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:14.445541 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:14.447192 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:14.447193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:14.448510 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:14.448513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:14.449474 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:14.449475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:14.449606 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:14.450231 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:14.450241 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:14.455788 (+ 5547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:14.465984 (+ 10196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:14.465997 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:14.466000 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:14.466347 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:14.466659 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:14.466660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:14.466661 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:14.466802 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:14.467350 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:14.467357 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:14.467555 (+ 198us) client_negotiation.cc:769] Sending connection context
0812 14:20:14.467663 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 14:20:14.467859 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2831,"thread_start_us":88,"threads_started":1}
I20260812 14:20:14.469161 23604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:14.440888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:45255 (local address 127.8.85.62:40297)
0812 14:20:14.443585 (+ 2697us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:14.443595 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:14.443832 (+ 237us) server_negotiation.cc:407] Connection header received
0812 14:20:14.444516 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:14.444521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:14.444611 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:14.444757 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:14.445674 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:14.447066 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:14.448614 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:14.449241 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:14.449354 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:14.463589 (+ 14235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:14.463597 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:14.463597 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:14.463616 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:14.465870 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:14.466454 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:14.466458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:14.466458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:14.466578 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:14.466894 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:14.466895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:14.466895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:14.467229 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:14.467317 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:14.468801 (+ 1484us) server_negotiation.cc:299] Negotiation successful
0812 14:20:14.468952 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2638,"thread_start_us":86,"threads_started":1}
I20260812 14:20:14.469961 23600 heartbeater.cc:347] Connected to a master server at 127.8.85.62:40297
I20260812 14:20:14.470070 23600 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:14.470281 23600 heartbeater.cc:511] Master 127.8.85.62:40297 requested a full tablet report, sending...
W20260812 14:20:14.471292 23600 heartbeater.cc:644] Failed to heartbeat to 127.8.85.62:40297 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 14:20:14.471369 23600 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:14.471544 23600 heartbeater.cc:511] Master 127.8.85.62:40297 requested a full tablet report, sending...
I20260812 14:20:14.471990 23600 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:14.472071 23600 heartbeater.cc:511] Master 127.8.85.62:40297 requested a full tablet report, sending...
W20260812 14:20:14.472419 23600 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:20:14.576524 23605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:14.576898 23605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:14.576998 23605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:14.578778 23605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:14.579010 23605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:14.579111 23605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:20:14.581380 23605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.85.20:52635/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:43679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=38723
--tserver_master_addrs=127.8.85.62:40297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:14.581851 23605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:14.582149 23605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:14.589490 23615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:14.589498 23612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:14.590452 23610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:14.591380 23605 server_base.cc:1034] running on GCE node
I20260812 14:20:14.591725 23605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:14.592015 23605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:14.595268 23605 hybrid_clock.cc:648] HybridClock initialized: now 1786544414595233 us; error 49 us; skew 500 ppm
Aug 12 14:20:14 dist-test-slave-5k54 krb5kdc[22237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:14.599033 23605 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:20:14.599634 23605 webserver.cc:460] Webserver started at http://127.8.85.2:38723/ using document root <none> and password file <none>
I20260812 14:20:14.599941 23605 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:14.600070 23605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 14:20:14 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:20:14.619166 23605 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 14:20:14.620249 23625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:14.620448 23605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:20:14.620528 23605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "eaa7052342a44f4f8a9948f96be57278"
format_stamp: "Formatted at 2026-08-12 14:20:13 on dist-test-slave-5k54"
server_key: "c3f2067c16068940ac8ec04ba48b664a"
server_key_iv: "9b8084890c33e9bd8ef5b4dcf5a103e5"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:14.621011 23605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:14.650012 23605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:14.652544 23605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:14.652776 23605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:14.653096 23605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:14.653535 23605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:14.653646 23605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:14.653728 23605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:14.653789 23605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:14.660697 23605 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:43679
I20260812 14:20:14.661345 23605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:20:14.663800 23761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:43679 every 8 connection(s)
I20260812 14:20:14.668377 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 23605
I20260812 14:20:14.669040 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.85.20:52635/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:40859
--local_ip_for_outbound_sockets=127.8.85.3
--tserver_master_addrs=127.8.85.62:40297
--webserver_port=45949
--webserver_interface=127.8.85.3
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:46703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.85.20:52635/kms with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:20:14 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:14.674053 23764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:14.664473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:40297 (local address 127.8.85.2:50369)
0812 14:20:14.664629 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:14.664671 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:14.665190 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:14.665492 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:14.665496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:14.665672 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:14.665871 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:14.665875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:14.666740 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:14.666741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:14.667614 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:14.667616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:14.668052 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:14.668054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:14.668209 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:14.668801 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:14.668822 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:14.670497 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:14.672306 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:14.672314 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:14.672316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:14.672627 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:14.672871 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:14.672872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:14.672872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:14.673008 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:14.673314 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:14.673322 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:14.673490 (+ 168us) client_negotiation.cc:769] Sending connection context
0812 14:20:14.673602 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 14:20:14.673761 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260812 14:20:14.674605 23604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:14.664461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:50369 (local address 127.8.85.62:40297)
0812 14:20:14.664508 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:14.664510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:14.664792 (+ 282us) server_negotiation.cc:407] Connection header received
0812 14:20:14.665306 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:14.665308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:14.665325 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:14.665379 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:14.666010 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:14.666638 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:14.667702 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:14.667926 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:14.668018 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:14.670608 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:14.670610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:14.670610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:14.670618 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:14.672200 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:14.672719 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:14.672720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:14.672721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:14.672788 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:14.673080 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:14.673081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:14.673081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:14.673220 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:14.673288 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:14.674370 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0812 14:20:14.674466 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:20:14.675326 23762 heartbeater.cc:347] Connected to a master server at 127.8.85.62:40297
I20260812 14:20:14.675462 23762 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:14.675671 23762 heartbeater.cc:511] Master 127.8.85.62:40297 requested a full tablet report, sending...
W20260812 14:20:14.676400 23762 heartbeater.cc:644] Failed to heartbeat to 127.8.85.62:40297 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 14:20:14.676474 23762 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:14.676542 23762 heartbeater.cc:511] Master 127.8.85.62:40297 requested a full tablet report, sending...
I20260812 14:20:14.676937 23762 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:14.677018 23762 heartbeater.cc:511] Master 127.8.85.62:40297 requested a full tablet report, sending...
W20260812 14:20:14.677383 23762 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:20:14.794559 23765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:14.794956 23765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:14.795140 23765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:14.797006 23765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:14.797250 23765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:14.797345 23765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:20:14.799751 23765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:46703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.85.20:52635/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:40859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=45949
--tserver_master_addrs=127.8.85.62:40297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:14.800243 23765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:14.800576 23765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:14.807387 23771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:14.807858 23770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:14.807981 23774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:14.808887 23765 server_base.cc:1034] running on GCE node
I20260812 14:20:14.809187 23765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:14.809443 23765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:14.814528 23765 hybrid_clock.cc:648] HybridClock initialized: now 1786544414813727 us; error 814 us; skew 500 ppm
Aug 12 14:20:14 dist-test-slave-5k54 krb5kdc[22237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:14.818065 23765 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:20:14.818807 23765 webserver.cc:460] Webserver started at http://127.8.85.3:45949/ using document root <none> and password file <none>
I20260812 14:20:14.819128 23765 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:14.819207 23765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 14:20:14 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:20:14.835913 23765 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 14:20:14.840502 23784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:14.840750 23765 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260812 14:20:14.840858 23765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "6b2112a43275483d869faac4d8176cf7"
format_stamp: "Formatted at 2026-08-12 14:20:13 on dist-test-slave-5k54"
server_key: "47daa2445394e6c3d3ae4a39522d8ecc"
server_key_iv: "8c6ab0a78790a09a4b943742a007e690"
server_key_version: "kuduclusterkey@0"
I20260812 14:20:14.841338 23765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:14.862106 23765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:14.864756 23765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:14.865031 23765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:14.865342 23765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:14.865809 23765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:14.865919 23765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:14.866000 23765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:14.866081 23765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:14.873118 23765 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:40859
I20260812 14:20:14.873610 23765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:20:14.875907 23919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:40859 every 8 connection(s)
I20260812 14:20:14.880481 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 23765
Aug 12 14:20:14 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:14.888424 23604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:14.877030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:54885 (local address 127.8.85.62:40297)
0812 14:20:14.877075 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:14.877077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:14.877327 (+ 250us) server_negotiation.cc:407] Connection header received
0812 14:20:14.877996 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:14.877997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:14.878010 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:14.878063 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:14.878825 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:14.879627 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:14.881255 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:14.881559 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:14.881607 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:14.884312 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:14.884314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:14.884315 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:14.884322 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:14.886267 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:14.886773 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:14.886774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:14.886774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:14.886837 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:14.887397 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:14.887398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:14.887398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:14.887776 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:14.887827 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:14.888219 (+ 392us) server_negotiation.cc:299] Negotiation successful
0812 14:20:14.888315 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:20:14.889147 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:14.876765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:40297 (local address 127.8.85.3:54885)
0812 14:20:14.877177 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:14.877238 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:14.877882 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:14.878244 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:14.878248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:14.878470 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:14.878726 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:14.878731 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:14.879736 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:14.879736 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:14.881151 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:14.881154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:14.881720 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:14.881721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:14.881861 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:14.882442 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:14.882454 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:14.884194 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:14.886392 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:14.886398 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:14.886400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:14.886684 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:14.886914 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:14.886914 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:14.886915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:14.887306 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:14.887884 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:14.887891 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:14.888059 (+ 168us) client_negotiation.cc:769] Sending connection context
0812 14:20:14.888754 (+ 695us) client_negotiation.cc:240] Negotiation successful
0812 14:20:14.888919 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":78,"threads_started":1}
I20260812 14:20:14.889631 23924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:14.880940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48576 (local address 127.8.85.62:40297)
0812 14:20:14.881111 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:14.881113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:14.881242 (+ 129us) server_negotiation.cc:407] Connection header received
0812 14:20:14.881344 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:14.881345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:14.881357 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:14.881402 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:14.882242 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:14.883249 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:14.884331 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:14.884740 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:14.884860 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:14.886680 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:14.886681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:14.886681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:14.886688 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:14.888266 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:14.888679 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:14.888680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:14.888681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:14.888738 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:14.888953 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:14.888954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:14.888955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:14.889071 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:14.889219 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:14.889446 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 14:20:14.889528 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":65,"threads_started":1}
I20260812 14:20:14.890105 23920 heartbeater.cc:347] Connected to a master server at 127.8.85.62:40297
I20260812 14:20:14.890199 23920 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:14.890405 23920 heartbeater.cc:511] Master 127.8.85.62:40297 requested a full tablet report, sending...
W20260812 14:20:14.891116 23920 heartbeater.cc:644] Failed to heartbeat to 127.8.85.62:40297 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 14:20:14.891188 23920 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:14.891247 23920 heartbeater.cc:511] Master 127.8.85.62:40297 requested a full tablet report, sending...
I20260812 14:20:14.891786 23920 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:14.891865 23920 heartbeater.cc:511] Master 127.8.85.62:40297 requested a full tablet report, sending...
W20260812 14:20:14.892194 23920 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 14:20:15 dist-test-slave-5k54 krb5kdc[22237](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786544415, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:15.472620 23600 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:15.472798 23600 heartbeater.cc:511] Master 127.8.85.62:40297 requested a full tablet report, sending...
I20260812 14:20:15.473237 23372 ts_manager.cc:194] Registered new tserver with Master: 32390f9513f2431b8f373b205da83ebe (127.8.85.1:42119)
I20260812 14:20:15.473964 23372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:45255
I20260812 14:20:15.677585 23762 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:15.677754 23762 heartbeater.cc:511] Master 127.8.85.62:40297 requested a full tablet report, sending...
I20260812 14:20:15.678236 23372 ts_manager.cc:194] Registered new tserver with Master: eaa7052342a44f4f8a9948f96be57278 (127.8.85.2:43679)
I20260812 14:20:15.678797 23372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:50369
Aug 12 14:20:15 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544415, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:20:15.892370 23920 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:15.892542 23920 heartbeater.cc:511] Master 127.8.85.62:40297 requested a full tablet report, sending...
I20260812 14:20:15.892904 23372 ts_manager.cc:194] Registered new tserver with Master: 6b2112a43275483d869faac4d8176cf7 (127.8.85.3:40859)
I20260812 14:20:15.893414 23372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:54885
I20260812 14:20:15.900990 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:20:15.909969 23946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:15.901953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48586 (local address 127.8.85.62:40297)
0812 14:20:15.902068 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:15.902070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:15.902128 (+ 58us) server_negotiation.cc:407] Connection header received
0812 14:20:15.902194 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:15.902194 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:15.902206 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:15.902276 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:15.902992 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:15.903783 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:15.904845 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:15.905138 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:15.905207 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:15.906725 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:15.906726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:15.906726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:15.906734 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:15.908628 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:15.909055 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:15.909057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:15.909057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:15.909127 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:15.909366 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:15.909367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:15.909368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:15.909518 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:15.909621 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:15.909773 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 14:20:15.909865 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":64,"threads_started":1}
I20260812 14:20:15.912231 23372 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:20:15.913978 23372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:20:15.922107 23954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:15.917337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:40859 (local address 127.0.0.1:40288)
0812 14:20:15.917978 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:15.917990 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:15.918043 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:15.918268 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:15.918271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:15.918273 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:15.918338 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:15.918340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:15.919617 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:15.919618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:15.920864 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:15.920867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:15.921822 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:15.921823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:15.921913 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:15.921929 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 14:20:15.921976 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 14:20:15.922023 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":65,"threads_started":1}
I20260812 14:20:15.922773 23957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:15.917545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40288 (local address 127.8.85.3:40859)
0812 14:20:15.917701 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:15.917707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:15.918102 (+ 395us) server_negotiation.cc:407] Connection header received
0812 14:20:15.918149 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:15.918152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:15.918173 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:15.918246 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:15.918405 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:15.919524 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:15.920975 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:15.921732 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:15.922562 (+ 830us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:15.922635 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:15.922663 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 14:20:15.922700 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":51,"threads_started":1}
I20260812 14:20:15.922765 23953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:15.917611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:42119 (local address 127.0.0.1:54336)
0812 14:20:15.918138 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:15.918146 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:15.918183 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:15.919034 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:15.919035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:15.919037 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:15.919072 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:15.919074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:15.920358 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:15.920359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:15.921478 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:15.921480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:15.922535 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:15.922536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:15.922591 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:15.922621 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 14:20:15.922654 (+ 33us) client_negotiation.cc:240] Negotiation successful
0812 14:20:15.922692 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":62,"threads_started":1}
I20260812 14:20:15.923177 23962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:15.917951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54336 (local address 127.8.85.1:42119)
0812 14:20:15.918866 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:15.918871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:15.918881 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:20:15.918916 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:15.918918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:15.918933 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:15.919135 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:15.919202 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:15.920261 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:15.921600 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:15.922419 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:15.922959 (+ 540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:15.923025 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:15.923050 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 14:20:15.923091 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":69,"threads_started":1}
I20260812 14:20:15.923766 23838 tablet_service.cc:1467] Processing CreateTablet for tablet 4b174d76da724d18b47471fc38cddb0d (DEFAULT_TABLE table=test-table [id=153bbcece0414f7cb47718f6fa7eecaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:20:15.923892 23509 tablet_service.cc:1467] Processing CreateTablet for tablet 4b174d76da724d18b47471fc38cddb0d (DEFAULT_TABLE table=test-table [id=153bbcece0414f7cb47718f6fa7eecaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:20:15.924162 23509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b174d76da724d18b47471fc38cddb0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:15.924162 23838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b174d76da724d18b47471fc38cddb0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:15.928309 23964 tablet_bootstrap.cc:492] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7: Bootstrap starting.
I20260812 14:20:15.929015 23964 tablet_bootstrap.cc:654] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:15.929378 23964 log.cc:824] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:15.929765 23963 tablet_bootstrap.cc:492] T 4b174d76da724d18b47471fc38cddb0d P 32390f9513f2431b8f373b205da83ebe: Bootstrap starting.
I20260812 14:20:15.930425 23963 tablet_bootstrap.cc:654] T 4b174d76da724d18b47471fc38cddb0d P 32390f9513f2431b8f373b205da83ebe: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:15.930804 23963 log.cc:824] T 4b174d76da724d18b47471fc38cddb0d P 32390f9513f2431b8f373b205da83ebe: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:15.931684 23964 tablet_bootstrap.cc:492] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7: No bootstrap required, opened a new log
I20260812 14:20:15.931808 23964 ts_tablet_manager.cc:1397] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 14:20:15.931843 23959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:15.917103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:43679 (local address 127.0.0.1:34228)
0812 14:20:15.917769 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:15.917816 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:15.917896 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:15.928071 (+ 10175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:15.928072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:15.928074 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:15.928119 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:15.928121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:15.929370 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:15.929371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:15.930487 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:15.930491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:15.931596 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:15.931597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:15.931659 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:15.931663 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:20:15.931714 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:20:15.931761 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":91,"threads_started":1}
I20260812 14:20:15.932111 23960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:15.917746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34228 (local address 127.8.85.2:43679)
0812 14:20:15.918474 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:15.918479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:15.918488 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:20:15.918520 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:15.918522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:15.918537 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:15.918605 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:15.928219 (+ 9614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:15.929260 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:15.930597 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:15.931402 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:15.931476 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:15.931533 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:15.931985 (+ 452us) server_negotiation.cc:299] Negotiation successful
0812 14:20:15.932035 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":51,"threads_started":1}
I20260812 14:20:15.933000 23687 tablet_service.cc:1467] Processing CreateTablet for tablet 4b174d76da724d18b47471fc38cddb0d (DEFAULT_TABLE table=test-table [id=153bbcece0414f7cb47718f6fa7eecaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:20:15.933194 23963 tablet_bootstrap.cc:492] T 4b174d76da724d18b47471fc38cddb0d P 32390f9513f2431b8f373b205da83ebe: No bootstrap required, opened a new log
I20260812 14:20:15.933259 23963 ts_tablet_manager.cc:1397] T 4b174d76da724d18b47471fc38cddb0d P 32390f9513f2431b8f373b205da83ebe: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 14:20:15.933429 23687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b174d76da724d18b47471fc38cddb0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:15.934715 23964 raft_consensus.cc:348] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa7052342a44f4f8a9948f96be57278" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43679 } } peers { permanent_uuid: "6b2112a43275483d869faac4d8176cf7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 40859 } } peers { permanent_uuid: "32390f9513f2431b8f373b205da83ebe" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42119 } }
I20260812 14:20:15.934895 23964 raft_consensus.cc:374] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:15.934933 23964 raft_consensus.cc:729] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b2112a43275483d869faac4d8176cf7, State: Initialized, Role: FOLLOWER
I20260812 14:20:15.935089 23964 consensus_queue.cc:260] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa7052342a44f4f8a9948f96be57278" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43679 } } peers { permanent_uuid: "6b2112a43275483d869faac4d8176cf7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 40859 } } peers { permanent_uuid: "32390f9513f2431b8f373b205da83ebe" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42119 } }
I20260812 14:20:15.935357 23964 ts_tablet_manager.cc:1428] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:20:15.935491 23920 heartbeater.cc:503] Master 127.8.85.62:40297 was elected leader, sending a full tablet report...
I20260812 14:20:15.935943 23963 raft_consensus.cc:348] T 4b174d76da724d18b47471fc38cddb0d P 32390f9513f2431b8f373b205da83ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa7052342a44f4f8a9948f96be57278" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43679 } } peers { permanent_uuid: "6b2112a43275483d869faac4d8176cf7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 40859 } } peers { permanent_uuid: "32390f9513f2431b8f373b205da83ebe" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42119 } }
I20260812 14:20:15.936128 23963 raft_consensus.cc:374] T 4b174d76da724d18b47471fc38cddb0d P 32390f9513f2431b8f373b205da83ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:15.936203 23963 raft_consensus.cc:729] T 4b174d76da724d18b47471fc38cddb0d P 32390f9513f2431b8f373b205da83ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32390f9513f2431b8f373b205da83ebe, State: Initialized, Role: FOLLOWER
I20260812 14:20:15.936345 23963 consensus_queue.cc:260] T 4b174d76da724d18b47471fc38cddb0d P 32390f9513f2431b8f373b205da83ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa7052342a44f4f8a9948f96be57278" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43679 } } peers { permanent_uuid: "6b2112a43275483d869faac4d8176cf7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 40859 } } peers { permanent_uuid: "32390f9513f2431b8f373b205da83ebe" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42119 } }
I20260812 14:20:15.936635 23963 ts_tablet_manager.cc:1428] T 4b174d76da724d18b47471fc38cddb0d P 32390f9513f2431b8f373b205da83ebe: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 14:20:15.936786 23600 heartbeater.cc:503] Master 127.8.85.62:40297 was elected leader, sending a full tablet report...
I20260812 14:20:15.937434 23970 tablet_bootstrap.cc:492] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278: Bootstrap starting.
I20260812 14:20:15.938050 23970 tablet_bootstrap.cc:654] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:15.938395 23970 log.cc:824] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:15.939142 23970 tablet_bootstrap.cc:492] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278: No bootstrap required, opened a new log
I20260812 14:20:15.939203 23970 ts_tablet_manager.cc:1397] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:20:15.941253 23970 raft_consensus.cc:348] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa7052342a44f4f8a9948f96be57278" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43679 } } peers { permanent_uuid: "6b2112a43275483d869faac4d8176cf7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 40859 } } peers { permanent_uuid: "32390f9513f2431b8f373b205da83ebe" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42119 } }
I20260812 14:20:15.941427 23970 raft_consensus.cc:374] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:15.941479 23970 raft_consensus.cc:729] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaa7052342a44f4f8a9948f96be57278, State: Initialized, Role: FOLLOWER
I20260812 14:20:15.941576 23970 consensus_queue.cc:260] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa7052342a44f4f8a9948f96be57278" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43679 } } peers { permanent_uuid: "6b2112a43275483d869faac4d8176cf7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 40859 } } peers { permanent_uuid: "32390f9513f2431b8f373b205da83ebe" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42119 } }
I20260812 14:20:15.941790 23970 ts_tablet_manager.cc:1428] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260812 14:20:15.941987 23601 tablet.cc:2369] T 4b174d76da724d18b47471fc38cddb0d P 32390f9513f2431b8f373b205da83ebe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:20:15.942315 23762 heartbeater.cc:503] Master 127.8.85.62:40297 was elected leader, sending a full tablet report...
Aug 12 14:20:16 dist-test-slave-5k54 krb5kdc[22237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544415, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for HTTP/127.8.85.20@KRBTEST.COM
I20260812 14:20:16.078783 23972 raft_consensus.cc:482] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:20:16.078939 23972 raft_consensus.cc:504] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa7052342a44f4f8a9948f96be57278" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43679 } } peers { permanent_uuid: "6b2112a43275483d869faac4d8176cf7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 40859 } } peers { permanent_uuid: "32390f9513f2431b8f373b205da83ebe" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42119 } }
I20260812 14:20:16.079355 23972 leader_election.cc:288] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b2112a43275483d869faac4d8176cf7 (127.8.85.3:40859), 32390f9513f2431b8f373b205da83ebe (127.8.85.1:42119)
I20260812 14:20:16.087025 23962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:16.079637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:57613 (local address 127.8.85.1:42119)
0812 14:20:16.079684 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:16.079686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:16.082150 (+ 2464us) server_negotiation.cc:407] Connection header received
0812 14:20:16.082255 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:16.082256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:16.082268 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:16.082298 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:16.082515 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:16.083247 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:16.085810 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:16.086400 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:16.086465 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:16.086477 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:16.086903 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 14:20:16.086949 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:20:16.087270 23975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:16.079782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:42119 (local address 127.8.85.2:57613)
0812 14:20:16.082077 (+ 2295us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:16.082089 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:16.082180 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:16.082361 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:16.082362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:16.082364 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:16.082435 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:16.082437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:16.083514 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:16.083515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:16.085517 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:16.085521 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:16.086737 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:16.086738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:16.086810 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:16.086814 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:20:16.087128 (+ 314us) client_negotiation.cc:240] Negotiation successful
0812 14:20:16.087172 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2266,"thread_start_us":98,"threads_started":1}
I20260812 14:20:16.087939 23538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b174d76da724d18b47471fc38cddb0d" candidate_uuid: "eaa7052342a44f4f8a9948f96be57278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32390f9513f2431b8f373b205da83ebe" is_pre_election: true
I20260812 14:20:16.088122 23538 raft_consensus.cc:2454] T 4b174d76da724d18b47471fc38cddb0d P 32390f9513f2431b8f373b205da83ebe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaa7052342a44f4f8a9948f96be57278 in term 0.
I20260812 14:20:16.088395 23628 leader_election.cc:302] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32390f9513f2431b8f373b205da83ebe, eaa7052342a44f4f8a9948f96be57278; no voters:
I20260812 14:20:16.088599 23957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:16.080431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:49279 (local address 127.8.85.3:40859)
0812 14:20:16.080477 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:16.080479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:16.080488 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:20:16.080519 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:16.080520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:16.080531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:16.080589 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:16.082712 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:16.083354 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:16.086553 (+ 3199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:16.087116 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:16.087163 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:16.087174 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:16.088454 (+ 1280us) server_negotiation.cc:299] Negotiation successful
0812 14:20:16.088503 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:20:16.088573 23972 raft_consensus.cc:2781] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:20:16.088653 23972 raft_consensus.cc:482] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:20:16.088678 23972 raft_consensus.cc:3037] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:16.089336 23976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:16.079530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:40859 (local address 127.8.85.2:49279)
0812 14:20:16.079941 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:16.079954 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:16.080010 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:16.080611 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:16.080613 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:16.080615 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:16.080700 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:16.080702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:16.083846 (+ 3144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:16.083847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:16.086456 (+ 2609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:16.086459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:16.087607 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:16.087608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:16.087682 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:16.087686 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:20:16.088678 (+ 992us) client_negotiation.cc:240] Negotiation successful
0812 14:20:16.088991 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":137,"threads_started":1}
I20260812 14:20:16.089793 23972 raft_consensus.cc:504] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa7052342a44f4f8a9948f96be57278" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43679 } } peers { permanent_uuid: "6b2112a43275483d869faac4d8176cf7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 40859 } } peers { permanent_uuid: "32390f9513f2431b8f373b205da83ebe" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42119 } }
I20260812 14:20:16.089944 23972 leader_election.cc:288] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [CANDIDATE]: Term 1 election: Requested vote from peers 6b2112a43275483d869faac4d8176cf7 (127.8.85.3:40859), 32390f9513f2431b8f373b205da83ebe (127.8.85.1:42119)
I20260812 14:20:16.090121 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b174d76da724d18b47471fc38cddb0d" candidate_uuid: "eaa7052342a44f4f8a9948f96be57278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b2112a43275483d869faac4d8176cf7" is_pre_election: true
I20260812 14:20:16.090276 23864 raft_consensus.cc:2454] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaa7052342a44f4f8a9948f96be57278 in term 0.
I20260812 14:20:16.090585 23538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b174d76da724d18b47471fc38cddb0d" candidate_uuid: "eaa7052342a44f4f8a9948f96be57278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32390f9513f2431b8f373b205da83ebe"
I20260812 14:20:16.090674 23538 raft_consensus.cc:3037] T 4b174d76da724d18b47471fc38cddb0d P 32390f9513f2431b8f373b205da83ebe [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:16.091544 23864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b174d76da724d18b47471fc38cddb0d" candidate_uuid: "eaa7052342a44f4f8a9948f96be57278" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b2112a43275483d869faac4d8176cf7"
I20260812 14:20:16.091617 23538 raft_consensus.cc:2454] T 4b174d76da724d18b47471fc38cddb0d P 32390f9513f2431b8f373b205da83ebe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaa7052342a44f4f8a9948f96be57278 in term 1.
I20260812 14:20:16.091626 23864 raft_consensus.cc:3037] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:16.091787 23628 leader_election.cc:302] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32390f9513f2431b8f373b205da83ebe, eaa7052342a44f4f8a9948f96be57278; no voters:
I20260812 14:20:16.091861 23972 raft_consensus.cc:2781] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:20:16.091996 23972 raft_consensus.cc:686] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [term 1 LEADER]: Becoming Leader. State: Replica: eaa7052342a44f4f8a9948f96be57278, State: Running, Role: LEADER
I20260812 14:20:16.092082 23972 consensus_queue.cc:237] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa7052342a44f4f8a9948f96be57278" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43679 } } peers { permanent_uuid: "6b2112a43275483d869faac4d8176cf7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 40859 } } peers { permanent_uuid: "32390f9513f2431b8f373b205da83ebe" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42119 } }
I20260812 14:20:16.092478 23864 raft_consensus.cc:2454] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaa7052342a44f4f8a9948f96be57278 in term 1.
I20260812 14:20:16.092983 23372 catalog_manager.cc:5495] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 reported cstate change: term changed from 0 to 1, leader changed from <none> to eaa7052342a44f4f8a9948f96be57278 (127.8.85.2). New cstate: current_term: 1 leader_uuid: "eaa7052342a44f4f8a9948f96be57278" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaa7052342a44f4f8a9948f96be57278" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b2112a43275483d869faac4d8176cf7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 40859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32390f9513f2431b8f373b205da83ebe" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42119 } health_report { overall_health: UNKNOWN } } }
I20260812 14:20:16.104957 23960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:16.101689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34240 (local address 127.8.85.2:43679)
0812 14:20:16.101731 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:16.101733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:16.101740 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:20:16.101777 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:16.101777 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:16.101800 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:16.101873 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:20:16.102033 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:16.102748 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:16.103894 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:16.104168 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:16.104235 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:16.104420 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:20:16.104558 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:20:16.104642 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:16.104862 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 14:20:16.104883 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:20:16.108693 23538 raft_consensus.cc:1261] T 4b174d76da724d18b47471fc38cddb0d P 32390f9513f2431b8f373b205da83ebe [term 1 FOLLOWER]: Refusing update from remote peer eaa7052342a44f4f8a9948f96be57278: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:20:16.109054 23864 raft_consensus.cc:1261] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7 [term 1 FOLLOWER]: Refusing update from remote peer eaa7052342a44f4f8a9948f96be57278: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:20:16.109521 23972 consensus_queue.cc:1035] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b2112a43275483d869faac4d8176cf7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 40859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:20:16.109982 23972 consensus_queue.cc:1035] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32390f9513f2431b8f373b205da83ebe" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 42119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:20:16.118037 23981 mvcc.cc:205] Tried to move back new op lower bound from 7317685928373481472 to 7317685928313630720. Current Snapshot: MvccSnapshot[applied={T|T < 7317685928373481472}]
I20260812 14:20:16.118182 23983 mvcc.cc:205] Tried to move back new op lower bound from 7317685928373481472 to 7317685928313630720. Current Snapshot: MvccSnapshot[applied={T|T < 7317685928373481472}]
I20260812 14:20:16.120736 23985 mvcc.cc:205] Tried to move back new op lower bound from 7317685928373481472 to 7317685928313630720. Current Snapshot: MvccSnapshot[applied={T|T < 7317685928373481472}]
I20260812 14:20:16.126498 23372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:20:16.126601 23372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:20:16.128105 23372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6590d5218e78421cbe3a81d0bd3e3a0c: Sending DeleteTablet for 3 replicas of tablet 4b174d76da724d18b47471fc38cddb0d
I20260812 14:20:16.128636 23687 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b174d76da724d18b47471fc38cddb0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:20:16 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:34228
I20260812 14:20:16.128651 23509 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b174d76da724d18b47471fc38cddb0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:20:16 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:54336
I20260812 14:20:16.128870 23838 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b174d76da724d18b47471fc38cddb0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:20:16 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:40288
I20260812 14:20:16.129611 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 23412
I20260812 14:20:16.135537 23991 tablet_replica.cc:330] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278: stopping tablet replica
I20260812 14:20:16.135713 23991 raft_consensus.cc:2229] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:20:16.135929 23991 raft_consensus.cc:2258] T 4b174d76da724d18b47471fc38cddb0d P eaa7052342a44f4f8a9948f96be57278 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 14:20:16.144555 23357 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:42119 (error 108)
I20260812 14:20:16.147531 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 23605
I20260812 14:20:16.152895 23959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:16.152728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:42119 (local address 127.0.0.1:54346)
0812 14:20:16.152789 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:16.152853 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:42119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 14:20:16.153024 23993 tablet_replica.cc:330] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7: stopping tablet replica
W20260812 14:20:16.153115 23357 catalog_manager.cc:4584] TS 32390f9513f2431b8f373b205da83ebe (127.8.85.1:42119): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4b174d76da724d18b47471fc38cddb0d: Network error: Client connection negotiation failed: client connection to 127.8.85.1:42119: connect: Connection refused (error 111)
I20260812 14:20:16.154009 23993 raft_consensus.cc:2229] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:20:16.155107 23993 raft_consensus.cc:2258] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:20:16.156095 23993 ts_tablet_manager.cc:1905] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:20:16.158010 23993 ts_tablet_manager.cc:1918] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:20:16.158144 23993 log.cc:1196] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/4b174d76da724d18b47471fc38cddb0d
I20260812 14:20:16.158526 23993 ts_tablet_manager.cc:1939] T 4b174d76da724d18b47471fc38cddb0d P 6b2112a43275483d869faac4d8176cf7: Deleting consensus metadata
I20260812 14:20:16.159054 23358 catalog_manager.cc:4841] TS 6b2112a43275483d869faac4d8176cf7 (127.8.85.3:40859): tablet 4b174d76da724d18b47471fc38cddb0d (table test-table [id=153bbcece0414f7cb47718f6fa7eecaf]) successfully deleted
I20260812 14:20:16.164196 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 23765
I20260812 14:20:16.166642 23959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:16.166510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:43679 (local address 127.0.0.1:34248)
0812 14:20:16.166573 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:16.166603 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.2:43679: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 14:20:16.180727 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 23335
I20260812 14:20:16.189662 8532 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 14:20:16.534073 8532 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 14:20:16.534199 8532 mini_ranger.cc:66] Stopping Ranger...
I20260812 14:20:16.879921 8532 mini_ranger.cc:68] Stopped Ranger
2026-08-12 14:20:16.880 UTC [22269] LOG: received smart shutdown request
2026-08-12 14:20:16.884 UTC [22269] LOG: background worker "logical replication launcher" (PID 22280) exited with exit code 1
2026-08-12 14:20:16.885 UTC [22274] LOG: shutting down
2026-08-12 14:20:17.058 UTC [22269] LOG: database system is shut down
2026-08-12T14:20:17Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48390 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:20:17 dist-test-slave-5k54 krb5kdc[24005](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:20:17 dist-test-slave-5k54 krb5kdc[24005](info): set up 2 sockets
Aug 12 14:20:17 dist-test-slave-5k54 krb5kdc[24005](info): commencing operation
krb5kdc: starting...
W20260812 14:20:18.289116 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:20:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:20:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:20:18.413901 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:37851
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:36901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:37851,127.8.85.61:38493,127.8.85.60:33365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:18.498503 24020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:18.498793 24020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:18.498838 24020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:18.500067 24020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:20:18.500131 24020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:18.500156 24020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:18.500185 24020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:20:18.500216 24020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:20:18.501729 24020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:37851,127.8.85.61:38493,127.8.85.60:33365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:37851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:18.502010 24020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:18.502243 24020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:18.507362 24026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:18.507362 24029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:18.507493 24027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:18.507555 24020 server_base.cc:1034] running on GCE node
I20260812 14:20:18.507943 24020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:18.508247 24020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:18.509395 24020 hybrid_clock.cc:648] HybridClock initialized: now 1786544418509376 us; error 40 us; skew 500 ppm
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:18.515096 24020 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:20:18.515638 24020 webserver.cc:460] Webserver started at http://127.8.85.62:41025/ using document root <none> and password file <none>
I20260812 14:20:18.515869 24020 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:18.515944 24020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:18.516099 24020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:20:18.517221 24020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "c5af20a2f6af45309ee4b2390d8ae8d3"
format_stamp: "Formatted at 2026-08-12 14:20:18 on dist-test-slave-5k54"
server_key: "9622c9c523945de03b105b9dbbc849b8"
server_key_iv: "73a95ac3a1f734e6c8f0e22b44124145"
server_key_version: "encryptionkey@0"
I20260812 14:20:18.517606 24020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "c5af20a2f6af45309ee4b2390d8ae8d3"
format_stamp: "Formatted at 2026-08-12 14:20:18 on dist-test-slave-5k54"
server_key: "9622c9c523945de03b105b9dbbc849b8"
server_key_iv: "73a95ac3a1f734e6c8f0e22b44124145"
server_key_version: "encryptionkey@0"
I20260812 14:20:18.518990 24020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:20:18.519809 24037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:18.519984 24020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:18.520090 24020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "c5af20a2f6af45309ee4b2390d8ae8d3"
format_stamp: "Formatted at 2026-08-12 14:20:18 on dist-test-slave-5k54"
server_key: "9622c9c523945de03b105b9dbbc849b8"
server_key_iv: "73a95ac3a1f734e6c8f0e22b44124145"
server_key_version: "encryptionkey@0"
I20260812 14:20:18.520166 24020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:18.531162 24020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:18.533324 24020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:18.533502 24020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:18.537770 24020 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:37851
I20260812 14:20:18.537807 24097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:37851 every 8 connection(s)
I20260812 14:20:18.538203 24020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:20:18.539081 24098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:18.539369 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 24020
I20260812 14:20:18.539538 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:20:18.539744 8532 external_mini_cluster.cc:1428] Setting key bc08e3ef09be77ca113a71b791e26392
I20260812 14:20:18.540876 24098 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.85.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.61@KRBTEST.COM" created.
I20260812 14:20:18.552960 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.541689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:37851 (local address 127.0.0.1:48750)
0812 14:20:18.541884 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.541922 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:18.542394 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:18.542790 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:18.542793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:18.543099 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:18.543258 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.543263 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.544925 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.544926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.545619 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.545621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.546422 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.546423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.546487 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.547062 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:18.547074 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:18.548741 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:18.551105 (+ 2364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.551109 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.551112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.551410 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.551735 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.551736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.551736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.551850 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.552397 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:18.552401 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:18.552477 (+ 76us) client_negotiation.cc:769] Sending connection context
0812 14:20:18.552572 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 14:20:18.552751 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":103,"threads_started":1}
I20260812 14:20:18.553360 24102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.541715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48750 (local address 127.8.85.62:37851)
0812 14:20:18.541895 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:18.541905 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:18.542023 (+ 118us) server_negotiation.cc:407] Connection header received
0812 14:20:18.542555 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:18.542558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:18.542582 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:18.542693 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:18.543378 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.544791 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.545707 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.546272 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.546409 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.548837 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:18.548845 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:18.548846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:18.548855 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:18.551006 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.551521 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.551525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.551525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.551636 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.551932 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.551933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.551934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.552230 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:18.553088 (+ 858us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:18.553144 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 14:20:18.553228 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260812 14:20:18.554334 24098 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:20:18.554729 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.554594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:38493 (local address 127.0.0.1:52632)
0812 14:20:18.554645 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.554696 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:38493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 14:20:18.554921 24041 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.85.61:38493: connect: Connection refused (error 111)
I20260812 14:20:18.555372 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.555274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:38493 (local address 127.0.0.1:52634)
0812 14:20:18.555325 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.555350 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:38493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 14:20:18.555527 24098 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.61:38493: Network error: Client connection negotiation failed: client connection to 127.8.85.61:38493: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.61.keytab.
Entry for principal kudu/127.8.85.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.61.keytab.
I20260812 14:20:18.568622 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.61:38493
--webserver_interface=127.8.85.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.61.keytab
--principal=kudu/127.8.85.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:36901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:37851,127.8.85.61:38493,127.8.85.60:33365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:20:18.604775 24098 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } attempt: 1
I20260812 14:20:18.605250 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.605115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:38493 (local address 127.0.0.1:52640)
0812 14:20:18.605176 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.605211 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:38493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 14:20:18.605574 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.605482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:38493 (local address 127.0.0.1:52648)
0812 14:20:18.605526 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.605551 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:38493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 14:20:18.605670 24098 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.61:38493: Network error: Client connection negotiation failed: client connection to 127.8.85.61:38493: connect: Connection refused (error 111)
W20260812 14:20:18.656347 24105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:18.656584 24105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:18.656622 24105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:18.657755 24105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:20:18.657823 24105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:18.657855 24105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:18.657881 24105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:20:18.657912 24105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:20:18.659443 24105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:37851,127.8.85.61:38493,127.8.85.60:33365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.61.keytab
--principal=kudu/127.8.85.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.61:38493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.85.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:18.659710 24105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:18.659950 24105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:18.665071 24110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:18.665133 24105 server_base.cc:1034] running on GCE node
W20260812 14:20:18.665073 24113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:18.665083 24111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:18.665566 24105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:18.665869 24105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:18.667028 24105 hybrid_clock.cc:648] HybridClock initialized: now 1786544418667004 us; error 37 us; skew 500 ppm
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:18.669625 24105 init.cc:377] Logged in from keytab as kudu/127.8.85.61@KRBTEST.COM (short username kudu)
I20260812 14:20:18.670137 24105 webserver.cc:460] Webserver started at http://127.8.85.61:36875/ using document root <none> and password file <none>
I20260812 14:20:18.670387 24105 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:18.670457 24105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:18.670610 24105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:20:18.671764 24105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/data/instance:
uuid: "c77072d5110049218274323a7da5ec0a"
format_stamp: "Formatted at 2026-08-12 14:20:18 on dist-test-slave-5k54"
server_key: "84c4a52e7655b5ee34d564429277f325"
server_key_iv: "787c5c3ee92a85088e934fe80d075eb5"
server_key_version: "encryptionkey@0"
I20260812 14:20:18.672145 24105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/wal/instance:
uuid: "c77072d5110049218274323a7da5ec0a"
format_stamp: "Formatted at 2026-08-12 14:20:18 on dist-test-slave-5k54"
server_key: "84c4a52e7655b5ee34d564429277f325"
server_key_iv: "787c5c3ee92a85088e934fe80d075eb5"
server_key_version: "encryptionkey@0"
I20260812 14:20:18.673471 24105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:18.673825 24098 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } attempt: 2
I20260812 14:20:18.674260 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.674111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:38493 (local address 127.0.0.1:52658)
0812 14:20:18.674160 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.674216 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:38493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 14:20:18.674291 24121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:18.674489 24105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:18.674598 24105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/wal
uuid: "c77072d5110049218274323a7da5ec0a"
format_stamp: "Formatted at 2026-08-12 14:20:18 on dist-test-slave-5k54"
server_key: "84c4a52e7655b5ee34d564429277f325"
server_key_iv: "787c5c3ee92a85088e934fe80d075eb5"
server_key_version: "encryptionkey@0"
I20260812 14:20:18.674674 24105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:18.674724 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.674580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:38493 (local address 127.0.0.1:52674)
0812 14:20:18.674632 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.674687 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:38493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 14:20:18.674888 24098 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.61:38493: Network error: Client connection negotiation failed: client connection to 127.8.85.61:38493: connect: Connection refused (error 111)
I20260812 14:20:18.687721 24105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:18.689564 24105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:18.689729 24105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:18.693825 24105 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.61:38493
I20260812 14:20:18.693890 24178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.61:38493 every 8 connection(s)
I20260812 14:20:18.694257 24105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/data/info.pb
I20260812 14:20:18.695124 24179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:18.696692 24179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.61@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:18.704003 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 24105
I20260812 14:20:18.704084 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-1/wal/instance
I20260812 14:20:18.704277 8532 external_mini_cluster.cc:1428] Setting key aeee8f045c7f9fc41eff4e68b85dd90f
I20260812 14:20:18.705224 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.697342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:37851 (local address 127.0.0.1:48766)
0812 14:20:18.697501 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.697541 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:18.697983 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:18.698186 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:18.698189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:18.698428 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:18.698619 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.698622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.699239 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.699240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.700095 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.700098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.700453 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.700453 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.700551 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.701045 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:18.701054 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:18.702404 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:18.703864 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.703870 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.703872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.704128 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.704351 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.704352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.704352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.704464 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.704753 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:18.704757 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:18.704872 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 14:20:18.704960 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 14:20:18.705074 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260812 14:20:18.705243 24102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.697389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.8.85.62:37851)
0812 14:20:18.697449 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:18.697451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:18.697624 (+ 173us) server_negotiation.cc:407] Connection header received
0812 14:20:18.698061 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:18.698062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:18.698072 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:18.698133 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:18.698687 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.699158 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.700171 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.700368 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.700432 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.702489 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:18.702490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:18.702491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:18.702497 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:18.703756 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.704194 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.704196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.704196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.704262 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.704535 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.704537 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.704537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.704684 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:18.704758 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:18.705005 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 14:20:18.705104 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:20:18.705906 24179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.61@KRBTEST.COM for kudu/127.8.85.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.85.60@KRBTEST.COM; defaulting to no policy
I20260812 14:20:18.713461 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.706381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:38493 (local address 127.0.0.1:52686)
0812 14:20:18.706431 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.706442 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:18.706481 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:18.706739 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:18.706740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:18.706878 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:18.706920 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.706922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.707855 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.707856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.708334 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.708336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.708815 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.708815 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.708888 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.709277 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:18.709282 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:18.710408 (+ 1126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:18.712440 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.712441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.712441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.712602 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.712831 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.712832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.712832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.712889 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.713222 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:18.713223 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:18.713269 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 14:20:18.713331 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 14:20:18.713402 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 14:20:18.713596 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.706455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52686 (local address 127.8.85.61:38493)
0812 14:20:18.706611 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:18.706614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:18.706631 (+ 17us) server_negotiation.cc:407] Connection header received
0812 14:20:18.706648 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:18.706649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:18.706660 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:18.706740 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:18.706996 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.707762 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.708438 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.708733 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.708817 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.710497 (+ 1680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:18.710505 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:18.710505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:18.710512 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:18.712350 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.712692 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.712696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.712696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.712759 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.712950 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.712951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.712951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.713121 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:18.713211 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:18.713391 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 14:20:18.713482 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260812 14:20:18.714202 24179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:20:18.714548 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.714432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:33365 (local address 127.0.0.1:59940)
0812 14:20:18.714478 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.714530 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 14:20:18.714696 24123 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
I20260812 14:20:18.715315 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.715224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:33365 (local address 127.0.0.1:59954)
0812 14:20:18.715267 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.715296 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 14:20:18.715471 24179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.60:33365: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.60.keytab.
Entry for principal kudu/127.8.85.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.60.keytab.
I20260812 14:20:18.732404 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.60:33365
--webserver_interface=127.8.85.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.60.keytab
--principal=kudu/127.8.85.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:36901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:37851,127.8.85.61:38493,127.8.85.60:33365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:20:18.764716 24179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } attempt: 1
I20260812 14:20:18.765306 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.765164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:33365 (local address 127.0.0.1:59956)
0812 14:20:18.765232 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.765275 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 14:20:18.765722 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.765623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:33365 (local address 127.0.0.1:59970)
0812 14:20:18.765669 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.765700 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 14:20:18.765821 24179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.60:33365: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
I20260812 14:20:18.766031 24098 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } attempt: 3
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for kudu/127.8.85.61@KRBTEST.COM
I20260812 14:20:18.773732 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.766299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:38493 (local address 127.0.0.1:52690)
0812 14:20:18.766367 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.766382 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:18.766442 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:18.766575 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:18.766576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:18.766716 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:18.766786 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.766788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.767770 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.767771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.768324 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.768327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.768809 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.768810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.768875 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.769399 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:18.769404 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:18.770970 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:18.772791 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.772792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.772792 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.772974 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.773185 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.773185 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.773186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.773240 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.773500 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:18.773500 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:18.773538 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 14:20:18.773590 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 14:20:18.773667 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 14:20:18.773804 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.766395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52690 (local address 127.8.85.61:38493)
0812 14:20:18.766438 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:18.766440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:18.766452 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:20:18.766490 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:18.766491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:18.766503 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:18.766569 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:18.766859 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.767665 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.768411 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.768725 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.768797 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.771073 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:18.771075 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:18.771076 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:18.771085 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:18.772706 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.773047 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.773048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.773048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.773113 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.773300 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.773301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.773301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.773437 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:18.773505 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:18.773636 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 14:20:18.773717 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:20:18.774261 24098 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:20:18.774609 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.774517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:33365 (local address 127.0.0.1:59974)
0812 14:20:18.774562 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.774590 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 14:20:18.774953 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.774840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:33365 (local address 127.0.0.1:59986)
0812 14:20:18.774911 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.774934 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 14:20:18.775041 24098 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.60:33365: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
I20260812 14:20:18.806197 24098 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } attempt: 1
I20260812 14:20:18.806773 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.806636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:33365 (local address 127.0.0.1:59990)
0812 14:20:18.806705 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.806744 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 14:20:18.807189 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.807084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:33365 (local address 127.0.0.1:59992)
0812 14:20:18.807135 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.807161 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 14:20:18.807291 24098 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.60:33365: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
W20260812 14:20:18.822324 24186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:18.822609 24186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:18.822647 24186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:18.823982 24186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:20:18.824056 24186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:18.824085 24186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:18.824108 24186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:20:18.824142 24186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:20:18.825712 24186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:37851,127.8.85.61:38493,127.8.85.60:33365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.60.keytab
--principal=kudu/127.8.85.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.60:33365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.85.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:18.825987 24186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:18.826256 24186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:18.831655 24194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:18.831620 24196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:18.831620 24193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:18.832008 24186 server_base.cc:1034] running on GCE node
I20260812 14:20:18.832254 24186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:18.832559 24186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:18.833711 24186 hybrid_clock.cc:648] HybridClock initialized: now 1786544418833685 us; error 44 us; skew 500 ppm
I20260812 14:20:18.833981 24179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } attempt: 2
I20260812 14:20:18.834475 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.834304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:33365 (local address 127.0.0.1:59994)
0812 14:20:18.834395 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.834436 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 14:20:18.834905 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.834774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:33365 (local address 127.0.0.1:60006)
0812 14:20:18.834846 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.834879 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 14:20:18.835034 24179 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.60:33365: Network error: Client connection negotiation failed: client connection to 127.8.85.60:33365: connect: Connection refused (error 111)
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:18.836874 24186 init.cc:377] Logged in from keytab as kudu/127.8.85.60@KRBTEST.COM (short username kudu)
I20260812 14:20:18.837286 24186 webserver.cc:460] Webserver started at http://127.8.85.60:34613/ using document root <none> and password file <none>
I20260812 14:20:18.837481 24186 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:18.837523 24186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:18.837618 24186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:20:18.838665 24186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/data/instance:
uuid: "af428f269d5f4b2ca4e63b349f0daa75"
format_stamp: "Formatted at 2026-08-12 14:20:18 on dist-test-slave-5k54"
server_key: "c247791e33b1ba57b4078d51d5255db9"
server_key_iv: "3df0e1ddce03030fd113fe94d6fc31c7"
server_key_version: "encryptionkey@0"
I20260812 14:20:18.838976 24186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/wal/instance:
uuid: "af428f269d5f4b2ca4e63b349f0daa75"
format_stamp: "Formatted at 2026-08-12 14:20:18 on dist-test-slave-5k54"
server_key: "c247791e33b1ba57b4078d51d5255db9"
server_key_iv: "3df0e1ddce03030fd113fe94d6fc31c7"
server_key_version: "encryptionkey@0"
I20260812 14:20:18.840391 24186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:20:18.841152 24206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:18.841440 24186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:20:18.841588 24186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/wal
uuid: "af428f269d5f4b2ca4e63b349f0daa75"
format_stamp: "Formatted at 2026-08-12 14:20:18 on dist-test-slave-5k54"
server_key: "c247791e33b1ba57b4078d51d5255db9"
server_key_iv: "3df0e1ddce03030fd113fe94d6fc31c7"
server_key_version: "encryptionkey@0"
I20260812 14:20:18.841670 24186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:18.857911 24186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:18.860008 24186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:18.860173 24186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:18.864184 24186 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.60:33365
I20260812 14:20:18.864235 24264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.60:33365 every 8 connection(s)
I20260812 14:20:18.864612 24186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/data/info.pb
I20260812 14:20:18.865581 24265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:18.867157 24265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:20:18.868021 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 24186
I20260812 14:20:18.868163 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786544263511571-8532-0/minicluster-data/master-2/wal/instance
I20260812 14:20:18.868350 8532 external_mini_cluster.cc:1428] Setting key e86d5334199b907d9e2da77bff0f7793
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.60@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.60@KRBTEST.COM
I20260812 14:20:18.879186 24102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.867985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48770 (local address 127.8.85.62:37851)
0812 14:20:18.868068 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:18.868070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:18.868335 (+ 265us) server_negotiation.cc:407] Connection header received
0812 14:20:18.869076 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:18.869078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:18.869090 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:18.869150 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:18.869822 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.870463 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.872114 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.872494 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.872562 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.875368 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:18.875370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:18.875370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:18.875378 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:18.877553 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.878001 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.878002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.878002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.878062 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.878406 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.878407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.878407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.878553 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:18.878626 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:18.878994 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 14:20:18.879089 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 14:20:18.879441 24267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.867965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:37851 (local address 127.0.0.1:48770)
0812 14:20:18.868179 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.868230 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:18.868963 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:18.869195 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:18.869200 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:18.869561 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:18.869708 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.869713 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.870558 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.870559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.872022 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.872024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.872624 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.872625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.872770 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.873436 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:18.873447 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:18.875272 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:18.877637 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.877644 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.877647 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.877916 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.878141 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.878142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.878142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.878317 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.878660 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:18.878666 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:18.878830 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 14:20:18.878947 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 14:20:18.879116 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":210,"thread_start_us":104,"threads_started":1}
I20260812 14:20:18.879751 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.868764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60010 (local address 127.8.85.60:33365)
0812 14:20:18.869084 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:18.869088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:18.869106 (+ 18us) server_negotiation.cc:407] Connection header received
0812 14:20:18.869189 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:18.869191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:18.869210 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:18.869298 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:18.870142 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.871313 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.872527 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.873137 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.874277 (+ 1140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.876786 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:18.876794 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:18.876794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:18.876801 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:18.878423 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.878806 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.878809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.878810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.878878 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.879107 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.879108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.879108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.879260 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:18.879443 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:18.879527 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 14:20:18.879617 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":80,"threads_started":1}
I20260812 14:20:18.880173 24265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:20:18.882413 24098 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } attempt: 2
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.60@KRBTEST.COM for kudu/127.8.85.61@KRBTEST.COM
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for kudu/127.8.85.60@KRBTEST.COM
I20260812 14:20:18.887346 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.880506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52692 (local address 127.8.85.61:38493)
0812 14:20:18.880547 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:18.880549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:18.880557 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:20:18.880583 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:18.880584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:18.880594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:18.880655 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:18.880923 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.881390 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.882043 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.882247 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.882322 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.884306 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:18.884308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:18.884308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:18.884316 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:18.885830 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.886253 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.886254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.886254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.886331 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.886763 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.886765 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.886765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.886912 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:18.886979 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:18.887177 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 14:20:18.887250 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:20:18.887673 24267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.880444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:38493 (local address 127.0.0.1:52692)
0812 14:20:18.880485 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.880494 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:18.880528 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:18.880660 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:18.880661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:18.880797 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:18.880840 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.880841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.881470 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.881471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.881946 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.881948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.882317 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.882318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.882392 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.882871 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:18.882876 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:18.884192 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:18.885926 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.885927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.885927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.886107 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.886581 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.886583 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.886583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.886681 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.887009 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:18.887010 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:18.887056 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 14:20:18.887480 (+ 424us) client_negotiation.cc:240] Negotiation successful
0812 14:20:18.887571 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 14:20:18.888113 24265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:20:18.889763 24102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.880599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48784 (local address 127.8.85.62:37851)
0812 14:20:18.880639 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:18.880641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:18.880649 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:20:18.880690 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:18.880691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:18.880700 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:18.880754 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:18.881202 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.881652 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.882325 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.882599 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.882662 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.885624 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:18.885625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:18.885626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:18.885633 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:18.887214 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.887637 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.887638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.887639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.887692 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.888034 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.888035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.888035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.888173 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:18.889311 (+ 1138us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:18.889564 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 14:20:18.889646 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:20:18.890866 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.882663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:33365 (local address 127.0.0.1:60016)
0812 14:20:18.882714 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.882724 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:18.882771 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:18.883167 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:18.883168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:18.883332 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:18.883372 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.883374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.884277 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.884278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.885083 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.885087 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.885638 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.885640 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.885705 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.886168 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:18.886173 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:18.887665 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:18.889664 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.889665 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.889666 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.889885 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.890110 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.890110 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.890111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.890173 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.890573 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:18.890574 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:18.890618 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 14:20:18.890690 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:20:18.890776 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 14:20:18.891330 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.882696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60016 (local address 127.8.85.60:33365)
0812 14:20:18.882728 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:18.882730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:18.882756 (+ 26us) server_negotiation.cc:407] Connection header received
0812 14:20:18.882825 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:18.882826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:18.882837 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:18.882900 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:18.883484 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.884187 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.885160 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.885530 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.885854 (+ 324us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.887758 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:18.887760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:18.887760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:18.887767 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:18.889571 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.889954 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.889955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.889955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.889998 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.890361 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.890362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.890362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.890496 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:18.890562 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:18.891171 (+ 609us) server_negotiation.cc:299] Negotiation successful
0812 14:20:18.891246 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.60@KRBTEST.COM for kudu/127.8.85.60@KRBTEST.COM
I20260812 14:20:18.893318 24098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3: Bootstrap starting.
I20260812 14:20:18.894162 24098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:18.894523 24098 log.cc:824] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3: Log is configured to *not* fsync() on all Append() calls
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.61@KRBTEST.COM
I20260812 14:20:18.895416 24098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3: No bootstrap required, opened a new log
I20260812 14:20:18.896314 24267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.888388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:33365 (local address 127.0.0.1:60020)
0812 14:20:18.888439 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.888453 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:18.888486 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:18.888767 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:18.888768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:18.888945 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:18.888969 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.888971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.890297 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.890298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.890958 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.890961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.891460 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.891461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.891510 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.891912 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:18.891917 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:18.893195 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:18.895073 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.895074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.895075 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.895237 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.895534 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.895534 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.895535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.895588 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.895963 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:18.895964 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:18.896011 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 14:20:18.896090 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 14:20:18.896196 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 14:20:18.896560 24270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.888439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60020 (local address 127.8.85.60:33365)
0812 14:20:18.888627 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:18.888629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:18.888635 (+ 6us) server_negotiation.cc:407] Connection header received
0812 14:20:18.888653 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:18.888654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:18.888663 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:18.888733 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:18.889062 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.890203 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.891041 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.891332 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.891394 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.893283 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:18.893285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:18.893285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:18.893292 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:18.894978 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.895329 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.895330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.895331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.895391 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.895662 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.895663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.895664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.895821 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:18.895970 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:18.896127 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 14:20:18.896252 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":169,"thread_start_us":116,"threads_started":1}
I20260812 14:20:18.898161 24265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75: Bootstrap starting.
I20260812 14:20:18.898396 24098 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } }
I20260812 14:20:18.898530 24098 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:18.898594 24098 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5af20a2f6af45309ee4b2390d8ae8d3, State: Initialized, Role: FOLLOWER
I20260812 14:20:18.898733 24098 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } }
I20260812 14:20:18.898892 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.890675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52702 (local address 127.8.85.61:38493)
0812 14:20:18.891023 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:18.891025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:18.891034 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:20:18.891051 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:18.891052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:18.891061 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:18.891111 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:18.891672 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.892106 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.892784 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.893050 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.893125 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.895748 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:18.895749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:18.895750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:18.895756 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:18.897425 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.897813 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.897814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.897814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.897876 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.898103 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.898104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.898104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.898233 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:18.898448 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:18.898680 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 14:20:18.898770 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326}
I20260812 14:20:18.898908 24265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:18.899149 24098 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:20:18.899209 24265 log.cc:824] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:18.900051 24265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75: No bootstrap required, opened a new log
I20260812 14:20:18.900419 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 14:20:18.901189 24288 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:20:18.901297 24288 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:20:18.901571 24272 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } } }
I20260812 14:20:18.901659 24272 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:20:18.902209 24265 raft_consensus.cc:348] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } }
I20260812 14:20:18.902377 24265 raft_consensus.cc:374] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:18.902423 24265 raft_consensus.cc:729] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af428f269d5f4b2ca4e63b349f0daa75, State: Initialized, Role: FOLLOWER
I20260812 14:20:18.902549 24265 consensus_queue.cc:260] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } }
I20260812 14:20:18.902875 24265 sys_catalog.cc:564] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:20:18.903008 24296 sys_catalog.cc:455] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } } }
I20260812 14:20:18.903088 24296 sys_catalog.cc:458] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:20:18.902029 24020 master_runner.cc:186] Error getting master registration for 127.8.85.61:38493: OK, instance_id { permanent_uuid: "c77072d5110049218274323a7da5ec0a" instance_seqno: 1786544418692639 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:20:18.905364 24020 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 14:20:18.905072 24308 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:20:18.905685 24308 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:20:18.907192 24186 master_runner.cc:186] Error getting master registration for 127.8.85.61:38493: OK, instance_id { permanent_uuid: "c77072d5110049218274323a7da5ec0a" instance_seqno: 1786544418692639 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:20:18.909564 24186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:20:18.913102 24102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.902451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48786 (local address 127.8.85.62:37851)
0812 14:20:18.902486 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:18.902488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:18.903243 (+ 755us) server_negotiation.cc:407] Connection header received
0812 14:20:18.903341 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:18.903343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:18.903368 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:18.903400 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:18.904216 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.904944 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.907057 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.907386 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.907448 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.908950 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:18.908952 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:18.908952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:18.908959 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:18.910621 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.911091 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.911092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.911094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.911150 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.911365 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.911366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.911366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.911562 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:18.911655 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:18.912934 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0812 14:20:18.913016 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:20:18.913746 24270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.902766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60028 (local address 127.8.85.60:33365)
0812 14:20:18.902912 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:18.902914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:18.905262 (+ 2348us) server_negotiation.cc:407] Connection header received
0812 14:20:18.905334 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:18.905335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:18.905346 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:18.905409 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:18.906098 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.907588 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.908589 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.908961 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.909002 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.910532 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:18.910533 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:18.910534 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:18.910541 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:18.912329 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.912740 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.912741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.912742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.912810 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.913039 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.913040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.913040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.913160 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:18.913252 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:18.913578 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 14:20:18.913662 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260812 14:20:18.915383 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.902088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52718 (local address 127.8.85.61:38493)
0812 14:20:18.902120 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:18.902122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:18.902401 (+ 279us) server_negotiation.cc:407] Connection header received
0812 14:20:18.902495 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:18.902496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:18.902508 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:18.902536 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:18.903320 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.904118 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.905119 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.905384 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.905431 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.911070 (+ 5639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:18.911071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:18.911072 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:18.911078 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:18.913928 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.914498 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.914499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.914499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.914558 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.914762 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.914763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.914763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.914902 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:18.914993 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:18.915207 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 14:20:18.915287 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:20:18.926213 24179 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } attempt: 3
Aug 12 14:20:18 dist-test-slave-5k54 krb5kdc[24005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544418, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.61@KRBTEST.COM for kudu/127.8.85.60@KRBTEST.COM
I20260812 14:20:18.934356 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.926488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:33365 (local address 127.0.0.1:60030)
0812 14:20:18.926534 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:18.926544 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:18.926579 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:18.926745 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:18.926746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:18.926919 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:18.926964 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.926966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.927941 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.927943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.928773 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:18.928775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.929249 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.929250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:18.929318 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.929809 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:18.929815 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:18.931372 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:18.933235 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.933236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.933236 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.933468 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.933687 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:18.933688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:18.933688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:18.933757 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:18.934031 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:18.934032 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:18.934078 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 14:20:18.934150 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:20:18.934243 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 14:20:18.934371 24270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:18.926564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60030 (local address 127.8.85.60:33365)
0812 14:20:18.926610 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:18.926612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:18.926622 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:20:18.926647 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:18.926648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:18.926658 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:18.926727 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:18.927048 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.927833 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.928859 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:18.929165 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:18.929238 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:18.931479 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:18.931481 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:18.931481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:18.931489 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:18.933143 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.933548 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.933549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.933549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.933610 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:18.933827 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:18.933828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:18.933828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:18.933959 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:18.934028 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:18.934192 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 14:20:18.934279 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:20:18.936523 24179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a: Bootstrap starting.
I20260812 14:20:18.937304 24179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:18.937621 24179 log.cc:824] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:18.938858 24179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a: No bootstrap required, opened a new log
I20260812 14:20:18.941087 24179 raft_consensus.cc:348] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } }
I20260812 14:20:18.941233 24179 raft_consensus.cc:374] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:18.941268 24179 raft_consensus.cc:729] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c77072d5110049218274323a7da5ec0a, State: Initialized, Role: FOLLOWER
I20260812 14:20:18.941466 24179 consensus_queue.cc:260] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } }
I20260812 14:20:18.941840 24179 sys_catalog.cc:564] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a [sys.catalog]: configured and running, proceeding with master startup.
W20260812 14:20:18.943785 24324 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:20:18.943889 24324 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:20:18.941808 24311 sys_catalog.cc:455] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } } }
I20260812 14:20:18.944111 24311 sys_catalog.cc:458] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:20:18.946365 24105 master_runner.cc:263] No leader master found from master c77072d5110049218274323a7da5ec0a
I20260812 14:20:18.946424 24105 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:20:18.981025 24296 raft_consensus.cc:482] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:20:18.981150 24296 raft_consensus.cc:504] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } }
I20260812 14:20:18.981377 24296 leader_election.cc:288] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5af20a2f6af45309ee4b2390d8ae8d3 (127.8.85.62:37851), c77072d5110049218274323a7da5ec0a (127.8.85.61:38493)
I20260812 14:20:18.981791 24069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af428f269d5f4b2ca4e63b349f0daa75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" is_pre_election: true
I20260812 14:20:18.981954 24069 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af428f269d5f4b2ca4e63b349f0daa75 in term 0.
I20260812 14:20:18.982126 24208 leader_election.cc:302] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af428f269d5f4b2ca4e63b349f0daa75, c5af20a2f6af45309ee4b2390d8ae8d3; no voters:
I20260812 14:20:18.982405 24296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:20:18.982465 24296 raft_consensus.cc:482] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:20:18.982486 24296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:18.983817 24151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af428f269d5f4b2ca4e63b349f0daa75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c77072d5110049218274323a7da5ec0a" is_pre_election: true
I20260812 14:20:18.983976 24151 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af428f269d5f4b2ca4e63b349f0daa75 in term 0.
I20260812 14:20:18.984253 24296 raft_consensus.cc:504] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } }
I20260812 14:20:18.984391 24296 leader_election.cc:288] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [CANDIDATE]: Term 1 election: Requested vote from peers c5af20a2f6af45309ee4b2390d8ae8d3 (127.8.85.62:37851), c77072d5110049218274323a7da5ec0a (127.8.85.61:38493)
I20260812 14:20:18.984623 24069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af428f269d5f4b2ca4e63b349f0daa75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3"
I20260812 14:20:18.984700 24069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:18.985635 24069 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af428f269d5f4b2ca4e63b349f0daa75 in term 1.
I20260812 14:20:18.985841 24208 leader_election.cc:302] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af428f269d5f4b2ca4e63b349f0daa75, c5af20a2f6af45309ee4b2390d8ae8d3; no voters:
I20260812 14:20:18.986109 24151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af428f269d5f4b2ca4e63b349f0daa75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c77072d5110049218274323a7da5ec0a"
I20260812 14:20:18.986186 24151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:18.987071 24151 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af428f269d5f4b2ca4e63b349f0daa75 in term 1.
I20260812 14:20:18.987334 24296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:20:18.987522 24296 raft_consensus.cc:686] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [term 1 LEADER]: Becoming Leader. State: Replica: af428f269d5f4b2ca4e63b349f0daa75, State: Running, Role: LEADER
I20260812 14:20:18.987614 24296 consensus_queue.cc:237] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } }
I20260812 14:20:18.987987 24296 sys_catalog.cc:455] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af428f269d5f4b2ca4e63b349f0daa75. Latest consensus state: current_term: 1 leader_uuid: "af428f269d5f4b2ca4e63b349f0daa75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } } }
I20260812 14:20:18.988076 24296 sys_catalog.cc:458] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [sys.catalog]: This master's current role is: LEADER
I20260812 14:20:18.988509 24329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:20:18.988737 24329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:20:18.991128 24151 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a [term 1 FOLLOWER]: Refusing update from remote peer af428f269d5f4b2ca4e63b349f0daa75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:20:18.992540 24296 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:20:18.992838 24069 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3 [term 1 FOLLOWER]: Refusing update from remote peer af428f269d5f4b2ca4e63b349f0daa75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:20:18.993867 24296 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:20:18.995198 24311 sys_catalog.cc:455] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader af428f269d5f4b2ca4e63b349f0daa75. Latest consensus state: current_term: 1 leader_uuid: "af428f269d5f4b2ca4e63b349f0daa75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } } }
I20260812 14:20:18.995286 24311 sys_catalog.cc:458] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:20:18.996399 24272 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af428f269d5f4b2ca4e63b349f0daa75. Latest consensus state: current_term: 1 leader_uuid: "af428f269d5f4b2ca4e63b349f0daa75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } } }
I20260812 14:20:18.996486 24272 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:20:18.997836 24296 sys_catalog.cc:455] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af428f269d5f4b2ca4e63b349f0daa75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } } }
I20260812 14:20:18.997918 24296 sys_catalog.cc:458] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [sys.catalog]: This master's current role is: LEADER
I20260812 14:20:18.998327 24311 sys_catalog.cc:455] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af428f269d5f4b2ca4e63b349f0daa75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } } }
I20260812 14:20:18.998405 24311 sys_catalog.cc:458] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:20:18.998672 24330 mvcc.cc:205] Tried to move back new op lower bound from 7317685940184854528 to 7317685940173815808. Current Snapshot: MvccSnapshot[applied={T|T < 7317685940184854528}]
I20260812 14:20:18.999333 24331 mvcc.cc:205] Tried to move back new op lower bound from 7317685940184854528 to 7317685940173815808. Current Snapshot: MvccSnapshot[applied={T|T < 7317685940184854528}]
I20260812 14:20:19.000159 24329 catalog_manager.cc:1347] Generated new cluster ID: ddbb0bcf4572413db18d5aa8ae3f79e7
I20260812 14:20:19.000211 24329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:20:19.010680 24327 sys_catalog.cc:455] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af428f269d5f4b2ca4e63b349f0daa75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } } }
I20260812 14:20:19.010794 24327 sys_catalog.cc:458] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75 [sys.catalog]: This master's current role is: LEADER
I20260812 14:20:19.011461 24272 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af428f269d5f4b2ca4e63b349f0daa75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5af20a2f6af45309ee4b2390d8ae8d3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 37851 } } peers { permanent_uuid: "c77072d5110049218274323a7da5ec0a" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 38493 } } peers { permanent_uuid: "af428f269d5f4b2ca4e63b349f0daa75" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 33365 } } }
I20260812 14:20:19.011554 24272 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:20:19.011844 24335 mvcc.cc:205] Tried to move back new op lower bound from 7317685940184854528 to 7317685940173815808. Current Snapshot: MvccSnapshot[applied={T|T < 7317685940184854528}]
I20260812 14:20:19.013929 24329 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:20:19.014807 24329 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:20:19.029197 24329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af428f269d5f4b2ca4e63b349f0daa75: Generated new TSK 0
I20260812 14:20:19.029465 24329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:20:19.901811 24288 catalog_manager.cc:1259] Loaded cluster ID: ddbb0bcf4572413db18d5aa8ae3f79e7
I20260812 14:20:19.901945 24288 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3: loading cluster ID for follower catalog manager: success
I20260812 14:20:19.902807 24288 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3: acquiring CA information for follower catalog manager: success
I20260812 14:20:19.903064 24288 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c5af20a2f6af45309ee4b2390d8ae8d3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:20:19.944377 24324 catalog_manager.cc:1259] Loaded cluster ID: ddbb0bcf4572413db18d5aa8ae3f79e7
I20260812 14:20:19.944492 24324 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a: loading cluster ID for follower catalog manager: success
I20260812 14:20:19.945363 24324 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a: acquiring CA information for follower catalog manager: success
I20260812 14:20:19.945647 24324 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c77072d5110049218274323a7da5ec0a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:20:20.085182 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 24020
I20260812 14:20:20.094985 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 24105
I20260812 14:20:20.100771 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 24186
2026-08-12T14:20:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2929 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:20:20 dist-test-slave-5k54 krb5kdc[24354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:20:20 dist-test-slave-5k54 krb5kdc[24354](info): set up 2 sockets
Aug 12 14:20:20 dist-test-slave-5k54 krb5kdc[24354](info): commencing operation
krb5kdc: starting...
W20260812 14:20:21.216426 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:20:21 dist-test-slave-5k54 krb5kdc[24354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:20:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:20:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:20:21.343055 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:40179
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:40179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:21.426973 24367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:21.427229 24367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:21.427268 24367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:21.428508 24367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:20:21.428588 24367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:21.428613 24367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:21.428633 24367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:20:21.428664 24367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:20:21.430193 24367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:40179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:40179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:21.430495 24367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:21.430735 24367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:21.435775 24376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:21.435794 24374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:21.435750 24373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:21.436045 24367 server_base.cc:1034] running on GCE node
I20260812 14:20:21.436280 24367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:21.436611 24367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:21.437790 24367 hybrid_clock.cc:648] HybridClock initialized: now 1786544421437767 us; error 52 us; skew 500 ppm
Aug 12 14:20:21 dist-test-slave-5k54 krb5kdc[24354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544421, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:21.440537 24367 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:20:21.441063 24367 webserver.cc:460] Webserver started at http://127.8.85.62:45659/ using document root <none> and password file <none>
I20260812 14:20:21.441330 24367 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:21.441406 24367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:21.441685 24367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:20:21.443058 24367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "2a8c9fcb08504fd9abe8ac5e7c31d221"
format_stamp: "Formatted at 2026-08-12 14:20:21 on dist-test-slave-5k54"
server_key: "fee199f0d23e56c39df5428f753cefb6"
server_key_iv: "31e29033a74f04ae0c63065a6f6f2413"
server_key_version: "encryptionkey@0"
I20260812 14:20:21.443465 24367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "2a8c9fcb08504fd9abe8ac5e7c31d221"
format_stamp: "Formatted at 2026-08-12 14:20:21 on dist-test-slave-5k54"
server_key: "fee199f0d23e56c39df5428f753cefb6"
server_key_iv: "31e29033a74f04ae0c63065a6f6f2413"
server_key_version: "encryptionkey@0"
I20260812 14:20:21.444941 24367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:21.445773 24384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:21.445976 24367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:21.446065 24367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "2a8c9fcb08504fd9abe8ac5e7c31d221"
format_stamp: "Formatted at 2026-08-12 14:20:21 on dist-test-slave-5k54"
server_key: "fee199f0d23e56c39df5428f753cefb6"
server_key_iv: "31e29033a74f04ae0c63065a6f6f2413"
server_key_version: "encryptionkey@0"
I20260812 14:20:21.446120 24367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:21.465739 24367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:21.468175 24367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:21.468292 24367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:21.472477 24367 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:40179
I20260812 14:20:21.472522 24444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:40179 every 8 connection(s)
I20260812 14:20:21.472903 24367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:20:21.473865 24445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:21.476292 24445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221: Bootstrap starting.
I20260812 14:20:21.476827 24445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:21.477087 24445 log.cc:824] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:21.477793 24445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221: No bootstrap required, opened a new log
I20260812 14:20:21.478566 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 24367
I20260812 14:20:21.478660 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:20:21.478850 8532 external_mini_cluster.cc:1428] Setting key d4cbb3daf8147ce9b7df68a55f16c59c
I20260812 14:20:21.480001 24445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8c9fcb08504fd9abe8ac5e7c31d221" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40179 } }
I20260812 14:20:21.480167 24445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:21.480217 24445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a8c9fcb08504fd9abe8ac5e7c31d221, State: Initialized, Role: FOLLOWER
I20260812 14:20:21.480409 24445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8c9fcb08504fd9abe8ac5e7c31d221" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40179 } }
I20260812 14:20:21.480505 24445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:20:21.480542 24445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:20:21.480584 24445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:21.481608 24445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8c9fcb08504fd9abe8ac5e7c31d221" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40179 } }
I20260812 14:20:21.481702 24445 leader_election.cc:302] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a8c9fcb08504fd9abe8ac5e7c31d221; no voters:
I20260812 14:20:21.481914 24445 leader_election.cc:288] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:20:21.481935 24451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:20:21.482050 24451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [term 1 LEADER]: Becoming Leader. State: Replica: 2a8c9fcb08504fd9abe8ac5e7c31d221, State: Running, Role: LEADER
I20260812 14:20:21.482187 24445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:20:21.482192 24451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8c9fcb08504fd9abe8ac5e7c31d221" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40179 } }
I20260812 14:20:21.482685 24453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a8c9fcb08504fd9abe8ac5e7c31d221. Latest consensus state: current_term: 1 leader_uuid: "2a8c9fcb08504fd9abe8ac5e7c31d221" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8c9fcb08504fd9abe8ac5e7c31d221" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40179 } } }
I20260812 14:20:21.482776 24453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [sys.catalog]: This master's current role is: LEADER
I20260812 14:20:21.483000 24452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a8c9fcb08504fd9abe8ac5e7c31d221" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a8c9fcb08504fd9abe8ac5e7c31d221" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40179 } } }
I20260812 14:20:21.483332 24452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221 [sys.catalog]: This master's current role is: LEADER
I20260812 14:20:21.483580 24458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:20:21.484076 24458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:20:21.485829 24458 catalog_manager.cc:1347] Generated new cluster ID: e6964d216324490aa948540d16de1d31
I20260812 14:20:21.485886 24458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:20:21 dist-test-slave-5k54 krb5kdc[24354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:21.491007 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:21.479393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60462 (local address 127.8.85.62:40179)
0812 14:20:21.479621 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:21.479629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:21.479654 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:20:21.479990 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:21.479998 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:21.480052 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:21.480197 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:21.481093 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:21.482264 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:21.483356 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:21.484079 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:21.484510 (+ 431us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:21.487070 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:21.487074 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:21.487074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:21.487087 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:21.489200 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:21.489617 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:21.489620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:21.489621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:21.489755 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:21.489970 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:21.489971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:21.489972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:21.490310 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:21.490506 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:21.490601 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 14:20:21.490780 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":137,"threads_started":1}
I20260812 14:20:21.492434 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:20:21.498335 24458 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:20:21.499061 24458 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:20:21.502408 24458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a8c9fcb08504fd9abe8ac5e7c31d221: Generated new TSK 0
I20260812 14:20:21.502589 24458 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:20:21.546540 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 24367
2026-08-12T14:20:21Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1444 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:20:21 dist-test-slave-5k54 krb5kdc[24475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:20:21 dist-test-slave-5k54 krb5kdc[24475](info): set up 2 sockets
Aug 12 14:20:21 dist-test-slave-5k54 krb5kdc[24475](info): commencing operation
krb5kdc: starting...
W20260812 14:20:22.662000 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:20:22 dist-test-slave-5k54 krb5kdc[24475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:20:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:20:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:20:22.788623 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:33235
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:33235
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:22.874034 24491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:22.874369 24491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:22.874455 24491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:22.875869 24491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:20:22.875944 24491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:22.875978 24491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:22.876006 24491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:20:22.876034 24491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:20:22.877506 24491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:33235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:33235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:22.877776 24491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:22.878031 24491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:22.882879 24501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:22.882864 24497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:22.882903 24496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:22.883109 24491 server_base.cc:1034] running on GCE node
I20260812 14:20:22.884243 24491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:22.884493 24491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:22.885660 24491 hybrid_clock.cc:648] HybridClock initialized: now 1786544422885647 us; error 45 us; skew 500 ppm
Aug 12 14:20:22 dist-test-slave-5k54 krb5kdc[24475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544422, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:22.888125 24491 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:20:22.888573 24491 webserver.cc:460] Webserver started at http://127.8.85.62:43817/ using document root <none> and password file <none>
I20260812 14:20:22.888774 24491 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:22.888814 24491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:22.888901 24491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:20:22.889879 24491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "b17f8b1fb57f44e7a38d482552e2ce3e"
format_stamp: "Formatted at 2026-08-12 14:20:22 on dist-test-slave-5k54"
server_key: "3f65701e554262378f45578b70895322"
server_key_iv: "2df3d3c7d27399fc5ae93576f3a4ed45"
server_key_version: "encryptionkey@0"
I20260812 14:20:22.890187 24491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "b17f8b1fb57f44e7a38d482552e2ce3e"
format_stamp: "Formatted at 2026-08-12 14:20:22 on dist-test-slave-5k54"
server_key: "3f65701e554262378f45578b70895322"
server_key_iv: "2df3d3c7d27399fc5ae93576f3a4ed45"
server_key_version: "encryptionkey@0"
I20260812 14:20:22.891558 24491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:20:22.892284 24509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:22.892490 24491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:22.892580 24491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "b17f8b1fb57f44e7a38d482552e2ce3e"
format_stamp: "Formatted at 2026-08-12 14:20:22 on dist-test-slave-5k54"
server_key: "3f65701e554262378f45578b70895322"
server_key_iv: "2df3d3c7d27399fc5ae93576f3a4ed45"
server_key_version: "encryptionkey@0"
I20260812 14:20:22.892637 24491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:22.903158 24491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:22.905579 24491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:22.905756 24491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:22.909794 24491 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:33235
I20260812 14:20:22.909828 24568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:33235 every 8 connection(s)
I20260812 14:20:22.910187 24491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:20:22.911221 24569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:22.913527 24569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: Bootstrap starting.
I20260812 14:20:22.913975 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 24491
I20260812 14:20:22.914096 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:20:22.914203 24569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:22.914328 8532 external_mini_cluster.cc:1428] Setting key 154f5a347f68481da56f7da15aa37908
I20260812 14:20:22.914557 24569 log.cc:824] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:22.915540 24569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: No bootstrap required, opened a new log
I20260812 14:20:22.917990 24569 raft_consensus.cc:348] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } }
I20260812 14:20:22.918164 24569 raft_consensus.cc:374] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:22.918200 24569 raft_consensus.cc:729] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17f8b1fb57f44e7a38d482552e2ce3e, State: Initialized, Role: FOLLOWER
I20260812 14:20:22.918313 24569 consensus_queue.cc:260] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } }
I20260812 14:20:22.918391 24569 raft_consensus.cc:388] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:20:22.918421 24569 raft_consensus.cc:482] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:20:22.918464 24569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:22.919059 24569 raft_consensus.cc:504] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } }
I20260812 14:20:22.919198 24569 leader_election.cc:302] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b17f8b1fb57f44e7a38d482552e2ce3e; no voters:
I20260812 14:20:22.919377 24569 leader_election.cc:288] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:20:22.919457 24574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:20:22.919597 24574 raft_consensus.cc:686] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 1 LEADER]: Becoming Leader. State: Replica: b17f8b1fb57f44e7a38d482552e2ce3e, State: Running, Role: LEADER
I20260812 14:20:22.919693 24569 sys_catalog.cc:564] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:20:22.919684 24574 consensus_queue.cc:237] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } }
I20260812 14:20:22.920069 24577 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b17f8b1fb57f44e7a38d482552e2ce3e. Latest consensus state: current_term: 1 leader_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } } }
I20260812 14:20:22.920203 24577 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [sys.catalog]: This master's current role is: LEADER
I20260812 14:20:22.920483 24583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:20:22.921000 24583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 14:20:22 dist-test-slave-5k54 krb5kdc[24475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:22.922554 24575 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } } }
I20260812 14:20:22.922634 24575 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [sys.catalog]: This master's current role is: LEADER
I20260812 14:20:22.923032 24583 catalog_manager.cc:1347] Generated new cluster ID: 060c4c96b5384723b62291ae4de3610e
I20260812 14:20:22.923079 24583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:20:22.925967 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:22.914849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36580 (local address 127.8.85.62:33235)
0812 14:20:22.915023 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:22.915032 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:22.915066 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:20:22.915669 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:22.915678 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:22.915737 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:22.915854 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:22.916658 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:22.917587 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:22.918670 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:22.919060 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:22.919179 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:22.922076 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:22.922081 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:22.922081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:22.922094 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:22.924156 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:22.924590 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:22.924593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:22.924594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:22.924696 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:22.924946 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:22.924947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:22.924947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:22.925228 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:22.925492 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:22.925627 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 14:20:22.925760 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260812 14:20:22.935503 24583 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:20:22.935990 24583 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:20:22.942266 24583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: Generated new TSK 0
I20260812 14:20:22.942473 24583 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:20:22.978317 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33235
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:23.062554 24602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:23.062750 24602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:23.062773 24602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:23.064014 24602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:23.064057 24602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:23.064083 24602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:20:23.065665 24602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:23.065932 24602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:23.066184 24602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:23.071230 24608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:23.071235 24611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:23.071238 24609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:23.071503 24602 server_base.cc:1034] running on GCE node
I20260812 14:20:23.071753 24602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:23.072014 24602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:23.073194 24602 hybrid_clock.cc:648] HybridClock initialized: now 1786544423073176 us; error 28 us; skew 500 ppm
Aug 12 14:20:23 dist-test-slave-5k54 krb5kdc[24475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544423, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:23.075881 24602 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:20:23.076395 24602 webserver.cc:460] Webserver started at http://127.8.85.1:43945/ using document root <none> and password file <none>
I20260812 14:20:23.076653 24602 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:23.076725 24602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:23.076882 24602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:20:23.077962 24602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "f27585b67c7146138f8ab173df86daf5"
format_stamp: "Formatted at 2026-08-12 14:20:23 on dist-test-slave-5k54"
server_key: "4176f2ee2f5fd5a9769ebca34c8094f6"
server_key_iv: "de0d291af584064b45c375bf2327e9d2"
server_key_version: "encryptionkey@0"
I20260812 14:20:23.078347 24602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "f27585b67c7146138f8ab173df86daf5"
format_stamp: "Formatted at 2026-08-12 14:20:23 on dist-test-slave-5k54"
server_key: "4176f2ee2f5fd5a9769ebca34c8094f6"
server_key_iv: "de0d291af584064b45c375bf2327e9d2"
server_key_version: "encryptionkey@0"
I20260812 14:20:23.079803 24602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:20:23.080557 24618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:23.080749 24602 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:20:23.080860 24602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "f27585b67c7146138f8ab173df86daf5"
format_stamp: "Formatted at 2026-08-12 14:20:23 on dist-test-slave-5k54"
server_key: "4176f2ee2f5fd5a9769ebca34c8094f6"
server_key_iv: "de0d291af584064b45c375bf2327e9d2"
server_key_version: "encryptionkey@0"
I20260812 14:20:23.080947 24602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:23.101537 24602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:23.103500 24602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:23.103674 24602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:23.103950 24602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:23.104285 24602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:23.104317 24602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:23.104341 24602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:23.104353 24602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:23.110153 24602 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:34593
I20260812 14:20:23.110204 24750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:34593 every 8 connection(s)
I20260812 14:20:23.110526 24602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:20:23.114320 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 24602
I20260812 14:20:23.114437 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:20:23.114614 8532 external_mini_cluster.cc:1428] Setting key 6b5cd8c40575ff835cb4968966aabedc
Aug 12 14:20:23 dist-test-slave-5k54 krb5kdc[24475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544423, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:23.121194 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.111304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:56817 (local address 127.8.85.62:33235)
0812 14:20:23.111370 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:23.111372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:23.111577 (+ 205us) server_negotiation.cc:407] Connection header received
0812 14:20:23.112052 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:23.112053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:23.112063 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:23.112126 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:23.113046 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.113547 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.114844 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.115180 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.115253 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.117958 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:23.117960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:23.117960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:23.117969 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:23.119608 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:23.120030 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:23.120032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:23.120032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:23.120098 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:23.120385 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:23.120386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:23.120386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:23.120601 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:23.120684 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:23.120976 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 14:20:23.121086 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:20:23.121358 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.111242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:56817)
0812 14:20:23.111406 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:23.111480 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:23.111966 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:23.112184 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:23.112187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:23.112759 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:23.112964 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.112969 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.113633 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.113634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.114734 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.114737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.115303 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.115304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.115484 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.116025 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:23.116035 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:23.117848 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:23.119697 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:23.119704 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:23.119706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:23.119948 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:23.120193 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:23.120194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:23.120195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:23.120312 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:23.120675 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:23.120683 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:23.120832 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 14:20:23.120944 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 14:20:23.121112 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260812 14:20:23.121896 24751 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33235
I20260812 14:20:23.122012 24751 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:23.122200 24751 heartbeater.cc:511] Master 127.8.85.62:33235 requested a full tablet report, sending...
I20260812 14:20:23.122803 24528 ts_manager.cc:194] Registered new tserver with Master: f27585b67c7146138f8ab173df86daf5 (127.8.85.1:34593)
I20260812 14:20:23.123524 24528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:56817
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:20:23.140246 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33235
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:23.223856 24756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:23.224069 24756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:23.224092 24756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:23.225370 24756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:23.225426 24756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:23.225453 24756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:20:23.227025 24756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:23.227284 24756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:23.227572 24756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:23.233232 24764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:23.233249 24761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:23.233251 24762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:23.233296 24756 server_base.cc:1034] running on GCE node
I20260812 14:20:23.233614 24756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:23.233886 24756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:23.235042 24756 hybrid_clock.cc:648] HybridClock initialized: now 1786544423235028 us; error 26 us; skew 500 ppm
Aug 12 14:20:23 dist-test-slave-5k54 krb5kdc[24475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544423, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:23.237558 24756 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:20:23.238054 24756 webserver.cc:460] Webserver started at http://127.8.85.2:43185/ using document root <none> and password file <none>
I20260812 14:20:23.238304 24756 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:23.238373 24756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:23.238523 24756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:20:23.239647 24756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "8f677095999642a1a2d10ba4066bde70"
format_stamp: "Formatted at 2026-08-12 14:20:23 on dist-test-slave-5k54"
server_key: "8bf7e1298d8ceceda27f3951d9074905"
server_key_iv: "2afc0a3f646b015b9bd763d17981f625"
server_key_version: "encryptionkey@0"
I20260812 14:20:23.240016 24756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "8f677095999642a1a2d10ba4066bde70"
format_stamp: "Formatted at 2026-08-12 14:20:23 on dist-test-slave-5k54"
server_key: "8bf7e1298d8ceceda27f3951d9074905"
server_key_iv: "2afc0a3f646b015b9bd763d17981f625"
server_key_version: "encryptionkey@0"
I20260812 14:20:23.241401 24756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 14:20:23.242182 24772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:23.242393 24756 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:23.242501 24756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "8f677095999642a1a2d10ba4066bde70"
format_stamp: "Formatted at 2026-08-12 14:20:23 on dist-test-slave-5k54"
server_key: "8bf7e1298d8ceceda27f3951d9074905"
server_key_iv: "2afc0a3f646b015b9bd763d17981f625"
server_key_version: "encryptionkey@0"
I20260812 14:20:23.242579 24756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:23.262027 24756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:23.264077 24756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:23.264245 24756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:23.264497 24756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:23.264835 24756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:23.264892 24756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:23.264955 24756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:23.264986 24756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:23.271116 24756 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:35887
I20260812 14:20:23.271169 24899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:35887 every 8 connection(s)
I20260812 14:20:23.271576 24756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:20:23.275648 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 24756
I20260812 14:20:23.275738 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:20:23.275972 8532 external_mini_cluster.cc:1428] Setting key a1ddcb03a7a6c6c78855137bf32d632f
Aug 12 14:20:23 dist-test-slave-5k54 krb5kdc[24475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544423, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:23.281453 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.272340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:40809 (local address 127.8.85.62:33235)
0812 14:20:23.272429 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:23.272431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:23.272617 (+ 186us) server_negotiation.cc:407] Connection header received
0812 14:20:23.273088 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:23.273089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:23.273100 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:23.273153 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:23.274030 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.274530 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.275544 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.275798 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.275869 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.278404 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:23.278405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:23.278406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:23.278413 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:23.279956 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:23.280425 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:23.280430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:23.280431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:23.280487 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:23.280759 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:23.280760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:23.280760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:23.280880 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:23.280959 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:23.281256 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 14:20:23.281351 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 14:20:23.281608 24902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.272316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:40809)
0812 14:20:23.272486 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:23.272523 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:23.273000 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:23.273203 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:23.273206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:23.273755 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:23.273953 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.273958 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.274615 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.274616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.275406 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.275408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.275917 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.275918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.276069 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.276678 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:23.276688 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:23.278290 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:23.280065 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:23.280072 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:23.280075 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:23.280306 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:23.280568 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:23.280569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:23.280569 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:23.280684 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:23.280962 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:23.280969 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:23.281108 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 14:20:23.281214 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 14:20:23.281375 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260812 14:20:23.282015 24900 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33235
I20260812 14:20:23.282135 24900 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:23.282317 24900 heartbeater.cc:511] Master 127.8.85.62:33235 requested a full tablet report, sending...
I20260812 14:20:23.282732 24528 ts_manager.cc:194] Registered new tserver with Master: 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887)
I20260812 14:20:23.283119 24528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:40809
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:20:23.303799 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33235
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:23.387684 24906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:23.387915 24906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:23.387956 24906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:23.389098 24906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:23.389168 24906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:23.389215 24906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:20:23.390707 24906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:23.390981 24906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:23.391222 24906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:23.396207 24917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:23.396196 24915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:23.396198 24914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:23.396808 24906 server_base.cc:1034] running on GCE node
I20260812 14:20:23.397023 24906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:23.397264 24906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:23.398412 24906 hybrid_clock.cc:648] HybridClock initialized: now 1786544423398410 us; error 41 us; skew 500 ppm
Aug 12 14:20:23 dist-test-slave-5k54 krb5kdc[24475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544423, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:23.400980 24906 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:20:23.401399 24906 webserver.cc:460] Webserver started at http://127.8.85.3:45947/ using document root <none> and password file <none>
I20260812 14:20:23.401676 24906 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:23.401757 24906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:23.401901 24906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:20:23.402940 24906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "81e60aaa3af74dbbb65d148f95028ab4"
format_stamp: "Formatted at 2026-08-12 14:20:23 on dist-test-slave-5k54"
server_key: "98b452e3c86d3e05156cf3264c24d663"
server_key_iv: "448f218adc57901e18183c5e95183308"
server_key_version: "encryptionkey@0"
I20260812 14:20:23.403322 24906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "81e60aaa3af74dbbb65d148f95028ab4"
format_stamp: "Formatted at 2026-08-12 14:20:23 on dist-test-slave-5k54"
server_key: "98b452e3c86d3e05156cf3264c24d663"
server_key_iv: "448f218adc57901e18183c5e95183308"
server_key_version: "encryptionkey@0"
I20260812 14:20:23.404714 24906 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:20:23.405438 24926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:23.405611 24906 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:23.405715 24906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "81e60aaa3af74dbbb65d148f95028ab4"
format_stamp: "Formatted at 2026-08-12 14:20:23 on dist-test-slave-5k54"
server_key: "98b452e3c86d3e05156cf3264c24d663"
server_key_iv: "448f218adc57901e18183c5e95183308"
server_key_version: "encryptionkey@0"
I20260812 14:20:23.405797 24906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:23.419822 24906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:23.422078 24906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:23.422245 24906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:23.422503 24906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:23.422838 24906 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:23.422873 24906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:23.422897 24906 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:23.422909 24906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:23.429284 24906 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:41981
I20260812 14:20:23.429332 25054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:41981 every 8 connection(s)
I20260812 14:20:23.429656 24906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:20:23 dist-test-slave-5k54 krb5kdc[24475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544423, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:23.438836 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.430385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:44549 (local address 127.8.85.62:33235)
0812 14:20:23.430454 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:23.430457 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:23.430679 (+ 222us) server_negotiation.cc:407] Connection header received
0812 14:20:23.431165 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:23.431166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:23.431176 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:23.431231 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:23.432235 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.432735 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.433743 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.433938 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.433995 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.436205 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:23.436206 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:23.436206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:23.436214 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:23.437620 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:23.438011 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:23.438012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:23.438013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:23.438064 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:23.438275 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:23.438293 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:23.438293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:23.438426 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:23.438476 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:23.438708 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 14:20:23.438768 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:20:23.439092 25057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.430370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:44549)
0812 14:20:23.430550 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:23.430588 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:23.431080 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:23.431287 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:23.431290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:23.431953 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:23.432155 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.432159 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.432847 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.432848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.433661 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.433663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.434021 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.434022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.434129 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.434612 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:23.434622 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:23.436116 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:23.437709 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:23.437715 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:23.437717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:23.437946 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:23.438124 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:23.438125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:23.438125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:23.438221 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:23.438480 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:23.438485 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:23.438599 (+ 114us) client_negotiation.cc:769] Sending connection context
0812 14:20:23.438682 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 14:20:23.438802 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260812 14:20:23.439371 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 24906
I20260812 14:20:23.439517 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:20:23.439707 8532 external_mini_cluster.cc:1428] Setting key b29e78c9e247142f3f46d90c660efc49
I20260812 14:20:23.439719 25055 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33235
I20260812 14:20:23.439821 25055 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:23.440053 25055 heartbeater.cc:511] Master 127.8.85.62:33235 requested a full tablet report, sending...
I20260812 14:20:23.440447 24528 ts_manager.cc:194] Registered new tserver with Master: 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981)
I20260812 14:20:23.440848 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:20:23.441004 24528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:44549
I20260812 14:20:23.448125 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.442374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36588 (local address 127.8.85.62:33235)
0812 14:20:23.442420 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:23.442423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:23.442548 (+ 125us) server_negotiation.cc:407] Connection header received
0812 14:20:23.442587 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:23.442588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:23.442596 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:23.442647 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:23.443219 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.443755 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.444440 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.444609 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.444657 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.445805 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:23.445807 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:23.445807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:23.445816 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:23.447206 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:23.447543 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:23.447544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:23.447544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:23.447592 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:23.447740 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:23.447740 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:23.447741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:23.447859 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:23.447929 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:23.448014 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 14:20:23.448070 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:20:23.449587 24528 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:20:23.451174 24528 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:20:23.457978 25067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.453733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:34593 (local address 127.0.0.1:42000)
0812 14:20:23.454090 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:23.454132 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:23.454209 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:23.454442 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:23.454445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:23.454448 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:23.454511 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.454513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.455696 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.455697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.456947 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.456949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.457689 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.457690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.457770 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.457783 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 14:20:23.457853 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 14:20:23.457909 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":76,"threads_started":1}
I20260812 14:20:23.458019 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.453877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42000 (local address 127.8.85.1:34593)
0812 14:20:23.454030 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:23.454035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:23.454201 (+ 166us) server_negotiation.cc:407] Connection header received
0812 14:20:23.454303 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:23.454306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:23.454321 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:23.454421 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:23.454609 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.455604 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.457047 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.457595 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.457689 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.457750 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:23.457894 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 14:20:23.457936 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260812 14:20:23.458285 25068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.453950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:41981 (local address 127.0.0.1:56572)
0812 14:20:23.454694 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:23.454703 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:23.454751 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:23.454973 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:23.454974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:23.454976 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:23.455017 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.455018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.456191 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.456192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.457101 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.457104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.458011 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.458012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.458097 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.458101 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:20:23.458152 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:20:23.458206 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"mutex_wait_us":25,"thread_start_us":78,"threads_started":1}
I20260812 14:20:23.458392 25070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.453925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42662 (local address 127.8.85.2:35887)
0812 14:20:23.454123 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:23.454128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:23.454320 (+ 192us) server_negotiation.cc:407] Connection header received
0812 14:20:23.454371 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:23.454373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:23.454388 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:23.454460 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:23.454655 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.455863 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.457117 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.457845 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.457918 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.457970 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:23.458251 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 14:20:23.458306 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":87,"threads_started":1}
I20260812 14:20:23.458402 25071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.453774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:35887 (local address 127.0.0.1:42662)
0812 14:20:23.454120 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:23.454132 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:23.454207 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:23.454537 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:23.454538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:23.454540 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:23.454591 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.454593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.455974 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.455975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.457049 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.457051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.458009 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.458010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.458096 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.458101 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:20:23.458152 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:20:23.458206 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":12,"thread_start_us":139,"threads_started":1}
I20260812 14:20:23.458670 25072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.454096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56572 (local address 127.8.85.3:41981)
0812 14:20:23.454274 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:23.454278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:23.454774 (+ 496us) server_negotiation.cc:407] Connection header received
0812 14:20:23.454822 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:23.454824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:23.454839 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:23.454960 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:23.455077 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.456092 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.457174 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.457852 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.457918 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.457970 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:23.458553 (+ 583us) server_negotiation.cc:299] Negotiation successful
0812 14:20:23.458598 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260812 14:20:23.459051 24671 tablet_service.cc:1467] Processing CreateTablet for tablet 082b841d2dcd4596b0e9bbd18eabbe75 (DEFAULT_TABLE table=test-table [id=712162dcff604585b419187b1feafc67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:20:23.459162 24824 tablet_service.cc:1467] Processing CreateTablet for tablet 082b841d2dcd4596b0e9bbd18eabbe75 (DEFAULT_TABLE table=test-table [id=712162dcff604585b419187b1feafc67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:20:23.459467 24671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 082b841d2dcd4596b0e9bbd18eabbe75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:23.459375 24978 tablet_service.cc:1467] Processing CreateTablet for tablet 082b841d2dcd4596b0e9bbd18eabbe75 (DEFAULT_TABLE table=test-table [id=712162dcff604585b419187b1feafc67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:20:23.459529 24824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 082b841d2dcd4596b0e9bbd18eabbe75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:23.459704 24978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 082b841d2dcd4596b0e9bbd18eabbe75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:23.461982 25073 tablet_bootstrap.cc:492] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: Bootstrap starting.
I20260812 14:20:23.462630 25073 tablet_bootstrap.cc:654] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:23.463016 25075 tablet_bootstrap.cc:492] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4: Bootstrap starting.
I20260812 14:20:23.463016 25074 tablet_bootstrap.cc:492] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70: Bootstrap starting.
I20260812 14:20:23.463254 25073 log.cc:824] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:23.463716 25074 tablet_bootstrap.cc:654] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:23.463716 25075 tablet_bootstrap.cc:654] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:23.464016 25073 tablet_bootstrap.cc:492] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: No bootstrap required, opened a new log
I20260812 14:20:23.464102 25073 ts_tablet_manager.cc:1397] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 14:20:23.464347 25074 log.cc:824] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:23.464347 25075 log.cc:824] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:23.465169 25074 tablet_bootstrap.cc:492] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70: No bootstrap required, opened a new log
I20260812 14:20:23.465245 25074 ts_tablet_manager.cc:1397] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:20:23.465420 25075 tablet_bootstrap.cc:492] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4: No bootstrap required, opened a new log
I20260812 14:20:23.465489 25075 ts_tablet_manager.cc:1397] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:20:23.466192 25073 raft_consensus.cc:348] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } }
I20260812 14:20:23.466357 25073 raft_consensus.cc:374] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:23.466401 25073 raft_consensus.cc:729] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f27585b67c7146138f8ab173df86daf5, State: Initialized, Role: FOLLOWER
I20260812 14:20:23.466513 25073 consensus_queue.cc:260] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } }
I20260812 14:20:23.466831 25073 ts_tablet_manager.cc:1428] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:20:23.466831 24751 heartbeater.cc:503] Master 127.8.85.62:33235 was elected leader, sending a full tablet report...
I20260812 14:20:23.467242 25074 raft_consensus.cc:348] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } }
I20260812 14:20:23.467453 25074 raft_consensus.cc:374] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:23.467501 25074 raft_consensus.cc:729] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f677095999642a1a2d10ba4066bde70, State: Initialized, Role: FOLLOWER
I20260812 14:20:23.467367 25075 raft_consensus.cc:348] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } }
I20260812 14:20:23.467579 25075 raft_consensus.cc:374] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:23.467626 25075 raft_consensus.cc:729] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81e60aaa3af74dbbb65d148f95028ab4, State: Initialized, Role: FOLLOWER
I20260812 14:20:23.467646 25074 consensus_queue.cc:260] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } }
I20260812 14:20:23.467730 25075 consensus_queue.cc:260] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } }
I20260812 14:20:23.467938 25074 ts_tablet_manager.cc:1428] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 14:20:23.468010 25075 ts_tablet_manager.cc:1428] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:20:23.468096 24900 heartbeater.cc:503] Master 127.8.85.62:33235 was elected leader, sending a full tablet report...
I20260812 14:20:23.468140 25055 heartbeater.cc:503] Master 127.8.85.62:33235 was elected leader, sending a full tablet report...
I20260812 14:20:23.469376 25081 raft_consensus.cc:482] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:20:23.469480 25081 raft_consensus.cc:504] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } }
I20260812 14:20:23.469822 25081 leader_election.cc:288] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887), 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981)
I20260812 14:20:23.473604 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.469954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:35887 (local address 127.8.85.1:50887)
0812 14:20:23.470039 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:23.470050 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:23.470096 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:23.470270 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:23.470271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:23.470272 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:23.470334 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.470335 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.471250 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.471251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.472588 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.472591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.473336 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.473337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.473426 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.473430 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:20:23.473487 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 14:20:23.473537 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":76}
I20260812 14:20:23.473609 25070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.469997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:50887 (local address 127.8.85.2:35887)
0812 14:20:23.470037 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:23.470039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:23.470154 (+ 115us) server_negotiation.cc:407] Connection header received
0812 14:20:23.470177 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:23.470178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:23.470189 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:23.470234 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:23.470402 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.471140 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.472682 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.473241 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.473321 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.473331 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:23.473491 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 14:20:23.473534 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:20:23.473664 25072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.470163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:50441 (local address 127.8.85.3:41981)
0812 14:20:23.470199 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:23.470201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:23.470209 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:20:23.470230 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:23.470230 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:23.470239 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:23.470293 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:23.470456 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.471136 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.472682 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.473241 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.473321 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.473331 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:23.473556 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 14:20:23.473596 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:20:23.473882 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.469984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:41981 (local address 127.8.85.1:50441)
0812 14:20:23.470117 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:23.470127 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:23.470168 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:23.470326 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:23.470326 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:23.470328 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:23.470382 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.470383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.471248 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.471250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.472588 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.472591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.473383 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.473384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.473468 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.473471 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:20:23.473741 (+ 270us) client_negotiation.cc:240] Negotiation successful
0812 14:20:23.473786 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":76,"thread_start_us":87,"threads_started":1}
I20260812 14:20:23.473953 24846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "082b841d2dcd4596b0e9bbd18eabbe75" candidate_uuid: "f27585b67c7146138f8ab173df86daf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f677095999642a1a2d10ba4066bde70" is_pre_election: true
I20260812 14:20:23.474107 24846 raft_consensus.cc:2454] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f27585b67c7146138f8ab173df86daf5 in term 0.
I20260812 14:20:23.474360 24622 leader_election.cc:302] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f677095999642a1a2d10ba4066bde70, f27585b67c7146138f8ab173df86daf5; no voters:
I20260812 14:20:23.474496 25081 raft_consensus.cc:2781] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:20:23.474491 25001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "082b841d2dcd4596b0e9bbd18eabbe75" candidate_uuid: "f27585b67c7146138f8ab173df86daf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81e60aaa3af74dbbb65d148f95028ab4" is_pre_election: true
I20260812 14:20:23.474563 25081 raft_consensus.cc:482] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:20:23.474610 25081 raft_consensus.cc:3037] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:23.474610 25001 raft_consensus.cc:2454] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f27585b67c7146138f8ab173df86daf5 in term 0.
I20260812 14:20:23.475297 25081 raft_consensus.cc:504] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } }
I20260812 14:20:23.475492 25081 leader_election.cc:288] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 1 election: Requested vote from peers 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887), 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981)
I20260812 14:20:23.475598 24846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "082b841d2dcd4596b0e9bbd18eabbe75" candidate_uuid: "f27585b67c7146138f8ab173df86daf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f677095999642a1a2d10ba4066bde70"
I20260812 14:20:23.475679 24846 raft_consensus.cc:3037] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:23.476300 24846 raft_consensus.cc:2454] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f27585b67c7146138f8ab173df86daf5 in term 1.
I20260812 14:20:23.476430 25001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "082b841d2dcd4596b0e9bbd18eabbe75" candidate_uuid: "f27585b67c7146138f8ab173df86daf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81e60aaa3af74dbbb65d148f95028ab4"
I20260812 14:20:23.476480 24622 leader_election.cc:302] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f677095999642a1a2d10ba4066bde70, f27585b67c7146138f8ab173df86daf5; no voters:
I20260812 14:20:23.476507 25001 raft_consensus.cc:3037] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:23.476583 25081 raft_consensus.cc:2781] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:20:23.476753 25081 raft_consensus.cc:686] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 1 LEADER]: Becoming Leader. State: Replica: f27585b67c7146138f8ab173df86daf5, State: Running, Role: LEADER
I20260812 14:20:23.476851 25081 consensus_queue.cc:237] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } }
I20260812 14:20:23.477366 25001 raft_consensus.cc:2454] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f27585b67c7146138f8ab173df86daf5 in term 1.
I20260812 14:20:23.477639 24526 catalog_manager.cc:5495] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to f27585b67c7146138f8ab173df86daf5 (127.8.85.1). New cstate: current_term: 1 leader_uuid: "f27585b67c7146138f8ab173df86daf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } health_report { overall_health: UNKNOWN } } }
I20260812 14:20:23.482563 25069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.480536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.8.85.1:34593)
0812 14:20:23.480592 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:23.480594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:23.480602 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:20:23.480631 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:23.480632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:23.480658 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:23.480715 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:20:23.480822 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.481274 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.481961 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.482129 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.482176 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.482309 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:20:23.482394 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:20:23.482451 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:23.482486 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 14:20:23.482503 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:20:23.484249 25001 raft_consensus.cc:1261] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4 [term 1 FOLLOWER]: Refusing update from remote peer f27585b67c7146138f8ab173df86daf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:20:23.484265 24846 raft_consensus.cc:1261] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70 [term 1 FOLLOWER]: Refusing update from remote peer f27585b67c7146138f8ab173df86daf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:20:23.484565 25085 consensus_queue.cc:1035] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:20:23.484727 25081 consensus_queue.cc:1035] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:20:23.487061 25088 mvcc.cc:205] Tried to move back new op lower bound from 7317685958588989440 to 7317685958561808384. Current Snapshot: MvccSnapshot[applied={T|T < 7317685958588989440}]
I20260812 14:20:23.487172 25090 mvcc.cc:205] Tried to move back new op lower bound from 7317685958588989440 to 7317685958561808384. Current Snapshot: MvccSnapshot[applied={T|T < 7317685958588989440}]
I20260812 14:20:23.489431 24525 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:20:23.489522 24525 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:20:23.490275 24525 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: Sending DeleteTablet for 3 replicas of tablet 082b841d2dcd4596b0e9bbd18eabbe75
I20260812 14:20:23.490573 24671 tablet_service.cc:1514] Processing DeleteTablet for tablet 082b841d2dcd4596b0e9bbd18eabbe75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:20:23 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:42000
I20260812 14:20:23.490655 24978 tablet_service.cc:1514] Processing DeleteTablet for tablet 082b841d2dcd4596b0e9bbd18eabbe75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:20:23 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:56572
I20260812 14:20:23.490655 24824 tablet_service.cc:1514] Processing DeleteTablet for tablet 082b841d2dcd4596b0e9bbd18eabbe75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:20:23 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:42662
I20260812 14:20:23.490777 25098 tablet_replica.cc:330] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: stopping tablet replica
I20260812 14:20:23.490860 25098 raft_consensus.cc:2229] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:20:23.490876 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 24602
I20260812 14:20:23.490984 25098 raft_consensus.cc:2258] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:20:23.496064 25100 tablet_replica.cc:330] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4: stopping tablet replica
I20260812 14:20:23.496145 25100 raft_consensus.cc:2229] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:20:23.496243 25100 raft_consensus.cc:2258] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:20:23.496423 25099 tablet_replica.cc:330] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70: stopping tablet replica
I20260812 14:20:23.496526 25099 raft_consensus.cc:2229] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:20:23.496613 25099 raft_consensus.cc:2258] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:20:23.497150 25099 ts_tablet_manager.cc:1905] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:20:23.497325 25100 ts_tablet_manager.cc:1905] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:20:23.498867 25100 ts_tablet_manager.cc:1918] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:20:23.498965 25100 log.cc:1196] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/082b841d2dcd4596b0e9bbd18eabbe75
I20260812 14:20:23.499161 25099 ts_tablet_manager.cc:1918] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:20:23.499239 25099 log.cc:1196] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/082b841d2dcd4596b0e9bbd18eabbe75
I20260812 14:20:23.499266 25100 ts_tablet_manager.cc:1939] T 082b841d2dcd4596b0e9bbd18eabbe75 P 81e60aaa3af74dbbb65d148f95028ab4: Deleting consensus metadata
I20260812 14:20:23.499547 25099 ts_tablet_manager.cc:1939] T 082b841d2dcd4596b0e9bbd18eabbe75 P 8f677095999642a1a2d10ba4066bde70: Deleting consensus metadata
I20260812 14:20:23.499782 24512 catalog_manager.cc:4841] TS 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981): tablet 082b841d2dcd4596b0e9bbd18eabbe75 (table test-table [id=712162dcff604585b419187b1feafc67]) successfully deleted
I20260812 14:20:23.499951 24513 catalog_manager.cc:4841] TS 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887): tablet 082b841d2dcd4596b0e9bbd18eabbe75 (table test-table [id=712162dcff604585b419187b1feafc67]) successfully deleted
W20260812 14:20:23.502326 24512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:34593 (error 108)
I20260812 14:20:23.502571 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 24756
I20260812 14:20:23.503715 25071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.502729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:34593 (local address 127.0.0.1:42004)
0812 14:20:23.503646 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:23.503688 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:34593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":891}
W20260812 14:20:23.504165 24512 catalog_manager.cc:4584] TS f27585b67c7146138f8ab173df86daf5 (127.8.85.1:34593): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 082b841d2dcd4596b0e9bbd18eabbe75: Network error: Client connection negotiation failed: client connection to 127.8.85.1:34593: connect: Connection refused (error 111)
I20260812 14:20:23.509727 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 24906
I20260812 14:20:23.516124 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 24491
I20260812 14:20:23.524119 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:33235
--webserver_interface=127.8.85.62
--webserver_port=43817
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:33235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:23.610832 25102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:23.611130 25102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:23.611167 25102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:23.612947 25102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:20:23.613026 25102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:23.613052 25102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:23.613080 25102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:20:23.613097 25102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:20:23.615636 25102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:33235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:33235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=43817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:23.615923 25102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:23.616168 25102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:23.625655 25113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:23.625694 25111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:23.625679 25110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:23.625836 25102 server_base.cc:1034] running on GCE node
I20260812 14:20:23.626086 25102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:23.626370 25102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:23.627488 25102 hybrid_clock.cc:648] HybridClock initialized: now 1786544423627469 us; error 39 us; skew 500 ppm
Aug 12 14:20:23 dist-test-slave-5k54 krb5kdc[24475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544423, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:23.638098 25102 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:20:23.638540 25102 webserver.cc:460] Webserver started at http://127.8.85.62:43817/ using document root <none> and password file <none>
I20260812 14:20:23.638751 25102 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:23.638804 25102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:23.640322 25102 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:23.640995 25120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:23.641194 25102 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:23.641317 25102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "b17f8b1fb57f44e7a38d482552e2ce3e"
format_stamp: "Formatted at 2026-08-12 14:20:22 on dist-test-slave-5k54"
server_key: "3f65701e554262378f45578b70895322"
server_key_iv: "2df3d3c7d27399fc5ae93576f3a4ed45"
server_key_version: "encryptionkey@0"
I20260812 14:20:23.641732 25102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:23.653083 25102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:23.654928 25102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:23.655037 25102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:23.660068 25102 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:33235
I20260812 14:20:23.660125 25179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:33235 every 8 connection(s)
I20260812 14:20:23.660549 25102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:20:23.662230 25180 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:20:23.662868 25180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: Bootstrap starting.
I20260812 14:20:23.669916 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 25102
I20260812 14:20:23.670231 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:34593
--local_ip_for_outbound_sockets=127.8.85.1
--tserver_master_addrs=127.8.85.62:33235
--webserver_port=43945
--webserver_interface=127.8.85.1
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:20:23.671492 25180 log.cc:824] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:23.673947 25180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:20:23.674248 25180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: Bootstrap complete.
I20260812 14:20:23.676745 25180 raft_consensus.cc:348] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } }
I20260812 14:20:23.676998 25180 raft_consensus.cc:729] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17f8b1fb57f44e7a38d482552e2ce3e, State: Initialized, Role: FOLLOWER
I20260812 14:20:23.677208 25180 consensus_queue.cc:260] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } }
I20260812 14:20:23.677309 25180 raft_consensus.cc:388] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:20:23.677371 25180 raft_consensus.cc:482] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:20:23.677429 25180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:20:23.678442 25180 raft_consensus.cc:504] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } }
I20260812 14:20:23.678565 25180 leader_election.cc:302] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b17f8b1fb57f44e7a38d482552e2ce3e; no voters:
I20260812 14:20:23.678766 25180 leader_election.cc:288] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 14:20:23.678800 25184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 2 FOLLOWER]: Leader election won for term 2
I20260812 14:20:23.678926 25184 raft_consensus.cc:686] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 2 LEADER]: Becoming Leader. State: Replica: b17f8b1fb57f44e7a38d482552e2ce3e, State: Running, Role: LEADER
I20260812 14:20:23.679052 25180 sys_catalog.cc:564] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:20:23.679088 25184 consensus_queue.cc:237] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } }
I20260812 14:20:23.679507 25186 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } } }
I20260812 14:20:23.679523 25187 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b17f8b1fb57f44e7a38d482552e2ce3e. Latest consensus state: current_term: 2 leader_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } } }
I20260812 14:20:23.679617 25186 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [sys.catalog]: This master's current role is: LEADER
I20260812 14:20:23.679636 25187 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [sys.catalog]: This master's current role is: LEADER
I20260812 14:20:23.680320 25195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:20:23.681123 25195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:20:23.681228 25195 catalog_manager.cc:1259] Loaded cluster ID: 060c4c96b5384723b62291ae4de3610e
I20260812 14:20:23.681262 25195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:20:23.681492 25195 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 14:20:23.686178 25195 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 14:20:23.688699 25195 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:20:23.688923 25195 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: Loaded TSK: 0
I20260812 14:20:23.689164 25195 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 14:20:23.756238 25183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:23.756469 25183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:23.756510 25183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:23.757669 25183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:23.757741 25183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:23.757788 25183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:20:23.759320 25183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:34593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=43945
--tserver_master_addrs=127.8.85.62:33235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:23.759626 25183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:23.759883 25183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:20:23.765012 25183 server_base.cc:1034] running on GCE node
W20260812 14:20:23.764948 25213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:23.765170 25216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:23.765240 25214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:23.765496 25183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:23.765774 25183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:23.766959 25183 hybrid_clock.cc:648] HybridClock initialized: now 1786544423766941 us; error 31 us; skew 500 ppm
Aug 12 14:20:23 dist-test-slave-5k54 krb5kdc[24475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544423, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:23.770021 25183 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:20:23.770512 25183 webserver.cc:460] Webserver started at http://127.8.85.1:43945/ using document root <none> and password file <none>
I20260812 14:20:23.770766 25183 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:23.770856 25183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:23.772424 25183 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:20:23.773154 25223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:23.773334 25183 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:20:23.773455 25183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "f27585b67c7146138f8ab173df86daf5"
format_stamp: "Formatted at 2026-08-12 14:20:23 on dist-test-slave-5k54"
server_key: "4176f2ee2f5fd5a9769ebca34c8094f6"
server_key_iv: "de0d291af584064b45c375bf2327e9d2"
server_key_version: "encryptionkey@0"
I20260812 14:20:23.773778 25183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:23.791630 25183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:23.794042 25183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:23.794211 25183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:23.794497 25183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:23.795003 25230 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 14:20:23.796705 25183 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 14:20:23.796762 25183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 14:20:23.796788 25183 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 14:20:23.797480 25183 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 14:20:23.797529 25183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 14:20:23.797577 25230 tablet_bootstrap.cc:492] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: Bootstrap starting.
I20260812 14:20:23.805480 25183 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:34593
I20260812 14:20:23.805559 25356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:34593 every 8 connection(s)
I20260812 14:20:23.806216 25183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:20:23.806903 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 25183
I20260812 14:20:23.807217 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:35887
--local_ip_for_outbound_sockets=127.8.85.2
--tserver_master_addrs=127.8.85.62:33235
--webserver_port=43185
--webserver_interface=127.8.85.2
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:20:23.809847 25230 log.cc:824] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:23.812978 25230 tablet_bootstrap.cc:492] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:20:23.813321 25230 tablet_bootstrap.cc:492] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: Bootstrap complete.
I20260812 14:20:23.814314 25230 ts_tablet_manager.cc:1397] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.003s
I20260812 14:20:23.815681 25230 raft_consensus.cc:348] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } }
I20260812 14:20:23.816015 25230 raft_consensus.cc:729] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f27585b67c7146138f8ab173df86daf5, State: Initialized, Role: FOLLOWER
I20260812 14:20:23.816187 25230 consensus_queue.cc:260] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } }
I20260812 14:20:23.816522 25230 ts_tablet_manager.cc:1428] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
Aug 12 14:20:23 dist-test-slave-5k54 krb5kdc[24475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544423, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:23.822216 25359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.807066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:36971)
0812 14:20:23.807264 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:23.807319 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:23.807959 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:23.810710 (+ 2751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:23.810715 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:23.811128 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:23.811402 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.811443 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.813267 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.813269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.814775 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.814779 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.815657 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.815659 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.815825 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.816452 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:23.816463 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:23.818330 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:23.820377 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:23.820387 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:23.820389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:23.820736 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:23.821024 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:23.821025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:23.821026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:23.821200 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:23.821565 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:23.821572 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:23.821725 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 14:20:23.821835 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 14:20:23.822009 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":94,"threads_started":1}
I20260812 14:20:23.822232 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.807419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:36971 (local address 127.8.85.62:33235)
0812 14:20:23.810205 (+ 2786us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:23.810215 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:23.810258 (+ 43us) server_negotiation.cc:407] Connection header received
0812 14:20:23.810432 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:23.810441 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:23.810506 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:23.810678 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:23.811585 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.813117 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.814894 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.815545 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.815672 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.818462 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:23.818467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:23.818467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:23.818478 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:23.820288 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:23.820832 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:23.820835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:23.820835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:23.820944 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:23.821276 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:23.821276 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:23.821276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:23.821494 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:23.821560 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:23.821962 (+ 402us) server_negotiation.cc:299] Negotiation successful
0812 14:20:23.822101 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2713,"thread_start_us":85,"threads_started":1}
I20260812 14:20:23.823177 25357 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33235
I20260812 14:20:23.823299 25357 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:23.823544 25357 heartbeater.cc:511] Master 127.8.85.62:33235 requested a full tablet report, sending...
I20260812 14:20:23.824283 25137 ts_manager.cc:194] Registered new tserver with Master: f27585b67c7146138f8ab173df86daf5 (127.8.85.1:34593)
I20260812 14:20:23.825460 25137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:36971
I20260812 14:20:23.826380 25357 heartbeater.cc:503] Master 127.8.85.62:33235 was elected leader, sending a full tablet report...
Aug 12 14:20:23 dist-test-slave-5k54 krb5kdc[24475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544423, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for kudu/127.8.85.1@KRBTEST.COM
I20260812 14:20:23.833339 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.824938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42012 (local address 127.8.85.1:34593)
0812 14:20:23.825151 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:23.825155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:23.825167 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:20:23.825252 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:23.825254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:23.825268 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:23.825340 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:23.825855 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.826838 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.827957 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.828235 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.828296 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.830400 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:23.830408 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:23.830408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:23.830415 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:23.832010 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:23.832429 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:23.832432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:23.832433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:23.832495 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:23.832711 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:23.832712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:23.832712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:23.832854 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:23.832953 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:23.833156 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 14:20:23.833247 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":120,"threads_started":1}
I20260812 14:20:23.833339 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.824862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:34593 (local address 127.0.0.1:42012)
0812 14:20:23.825058 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:23.825100 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:23.825165 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:23.825351 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:23.825354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:23.825684 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:23.825740 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.825742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.826917 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.826918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.827874 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.827878 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.828316 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.828320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.828400 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.828839 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:23.828844 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:23.830289 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:23.832119 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:23.832127 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:23.832129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:23.832333 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:23.832576 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:23.832577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:23.832578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:23.832644 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:23.832946 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:23.832953 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:23.833007 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 14:20:23.833116 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 14:20:23.833241 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":104,"threads_started":1}
I20260812 14:20:23.833772 25278 tablet_service.cc:1514] Processing DeleteTablet for tablet 082b841d2dcd4596b0e9bbd18eabbe75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:20:23 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:42012
I20260812 14:20:23.834311 25366 tablet_replica.cc:330] stopping tablet replica
I20260812 14:20:23.834427 25366 raft_consensus.cc:2229] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:20:23.834496 25366 raft_consensus.cc:2258] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:20:23.834802 25366 ts_tablet_manager.cc:1905] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:20:23.837173 25366 ts_tablet_manager.cc:1918] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:20:23.837258 25366 log.cc:1196] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal/wals/082b841d2dcd4596b0e9bbd18eabbe75
I20260812 14:20:23.837500 25366 ts_tablet_manager.cc:1939] T 082b841d2dcd4596b0e9bbd18eabbe75 P f27585b67c7146138f8ab173df86daf5: Deleting consensus metadata
I20260812 14:20:23.838027 25123 catalog_manager.cc:4841] TS f27585b67c7146138f8ab173df86daf5 (127.8.85.1:34593): tablet 082b841d2dcd4596b0e9bbd18eabbe75 (table test-table [id=712162dcff604585b419187b1feafc67]) successfully deleted
W20260812 14:20:23.898536 25361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:23.898736 25361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:23.898758 25361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:23.899936 25361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:23.899978 25361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:23.900005 25361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:20:23.901440 25361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:35887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=43185
--tserver_master_addrs=127.8.85.62:33235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:23.901630 25361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:23.901877 25361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:23.906957 25371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:23.906967 25372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:23.906970 25374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:23.907250 25361 server_base.cc:1034] running on GCE node
I20260812 14:20:23.907564 25361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:23.907768 25361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:23.908908 25361 hybrid_clock.cc:648] HybridClock initialized: now 1786544423908897 us; error 29 us; skew 500 ppm
Aug 12 14:20:23 dist-test-slave-5k54 krb5kdc[24475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544423, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:23.911373 25361 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:20:23.911866 25361 webserver.cc:460] Webserver started at http://127.8.85.2:43185/ using document root <none> and password file <none>
I20260812 14:20:23.912110 25361 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:23.912200 25361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:23.913575 25361 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:20:23.914289 25382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:23.914481 25361 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:20:23.914587 25361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "8f677095999642a1a2d10ba4066bde70"
format_stamp: "Formatted at 2026-08-12 14:20:23 on dist-test-slave-5k54"
server_key: "8bf7e1298d8ceceda27f3951d9074905"
server_key_iv: "2afc0a3f646b015b9bd763d17981f625"
server_key_version: "encryptionkey@0"
I20260812 14:20:23.914961 25361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:23.932550 25361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:23.934659 25361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:23.934829 25361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:23.935101 25361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:23.935477 25361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:23.935549 25361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:23.935591 25361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:23.935619 25361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:23.941548 25361 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:35887
I20260812 14:20:23.941636 25511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:35887 every 8 connection(s)
I20260812 14:20:23.942001 25361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:20:23.944795 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 25361
I20260812 14:20:23.945086 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:41981
--local_ip_for_outbound_sockets=127.8.85.3
--tserver_master_addrs=127.8.85.62:33235
--webserver_port=45947
--webserver_interface=127.8.85.3
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:20:23 dist-test-slave-5k54 krb5kdc[24475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544423, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:23.953213 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.942842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:45493 (local address 127.8.85.62:33235)
0812 14:20:23.942917 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:23.942920 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:23.943169 (+ 249us) server_negotiation.cc:407] Connection header received
0812 14:20:23.943768 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:23.943770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:23.943783 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:23.943839 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:23.944941 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.945677 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.947156 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.947386 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.947514 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.950138 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:23.950139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:23.950140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:23.950147 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:23.951639 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:23.952194 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:23.952195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:23.952195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:23.952244 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:23.952540 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:23.952540 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:23.952540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:23.952637 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:23.952685 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:23.953078 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 14:20:23.953146 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 14:20:23.953538 25516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:23.942832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:45493)
0812 14:20:23.943040 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:23.943077 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:23.943646 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:23.943919 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:23.943923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:23.944494 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:23.944812 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.944819 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.945780 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.945781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.947058 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:23.947061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:23.947525 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:23.947526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:23.947652 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:23.948216 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:23.948225 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:23.950038 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:23.951722 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:23.951731 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:23.951734 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:23.952108 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:23.952314 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:23.952314 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:23.952315 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:23.952468 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:23.952708 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:23.952714 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:23.952927 (+ 213us) client_negotiation.cc:769] Sending connection context
0812 14:20:23.953063 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 14:20:23.953255 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":124,"threads_started":1}
I20260812 14:20:23.954121 25512 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33235
I20260812 14:20:23.954237 25512 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:23.954452 25512 heartbeater.cc:511] Master 127.8.85.62:33235 requested a full tablet report, sending...
I20260812 14:20:23.954900 25137 ts_manager.cc:194] Registered new tserver with Master: 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887)
I20260812 14:20:23.955281 25137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:45493
W20260812 14:20:24.031100 25517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:24.031303 25517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:24.031327 25517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:24.032508 25517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:24.032553 25517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:24.032580 25517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:20:24.034029 25517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:41981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=45947
--tserver_master_addrs=127.8.85.62:33235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:24.034276 25517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:24.034528 25517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:24.039662 25526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:24.039688 25524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:24.039656 25523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:24.039752 25517 server_base.cc:1034] running on GCE node
I20260812 14:20:24.040094 25517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:24.040338 25517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:24.041515 25517 hybrid_clock.cc:648] HybridClock initialized: now 1786544424041501 us; error 46 us; skew 500 ppm
Aug 12 14:20:24 dist-test-slave-5k54 krb5kdc[24475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544424, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:24.044319 25517 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:20:24.044865 25517 webserver.cc:460] Webserver started at http://127.8.85.3:45947/ using document root <none> and password file <none>
I20260812 14:20:24.045054 25517 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:24.045140 25517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:24.046483 25517 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:20:24.047078 25535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:24.047254 25517 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:20:24.047341 25517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "81e60aaa3af74dbbb65d148f95028ab4"
format_stamp: "Formatted at 2026-08-12 14:20:23 on dist-test-slave-5k54"
server_key: "98b452e3c86d3e05156cf3264c24d663"
server_key_iv: "448f218adc57901e18183c5e95183308"
server_key_version: "encryptionkey@0"
I20260812 14:20:24.047719 25517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:24.060426 25517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:24.062206 25517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:24.062315 25517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:24.062493 25517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:24.062803 25517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:24.062836 25517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:24.062860 25517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:24.062916 25517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:24.068677 25517 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:41981
I20260812 14:20:24.068760 25664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:41981 every 8 connection(s)
I20260812 14:20:24.069092 25517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:20:24.071333 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 25517
Aug 12 14:20:24 dist-test-slave-5k54 krb5kdc[24475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544424, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:24.078995 25669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.071810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36590 (local address 127.8.85.62:33235)
0812 14:20:24.071939 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:24.071941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:24.071974 (+ 33us) server_negotiation.cc:407] Connection header received
0812 14:20:24.072042 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:24.072042 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:24.072053 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:24.072116 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:24.072766 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.073212 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.074189 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.074485 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.074537 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:24.075790 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:24.075792 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:24.075792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:24.075800 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:24.077735 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:24.078105 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:24.078107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:24.078107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:24.078167 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:24.078421 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:24.078422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:24.078422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:24.078559 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:24.078725 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:24.078782 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 14:20:24.078898 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":58,"threads_started":1}
I20260812 14:20:24.079597 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.069821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:45843 (local address 127.8.85.62:33235)
0812 14:20:24.069926 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:24.069928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:24.070099 (+ 171us) server_negotiation.cc:407] Connection header received
0812 14:20:24.070562 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:24.070563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:24.070573 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:24.070624 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:24.071583 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.072273 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.073516 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.073695 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.073748 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:24.076385 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:24.076386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:24.076386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:24.076392 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:24.077838 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:24.078299 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:24.078301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:24.078301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:24.078352 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:24.078649 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:24.078650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:24.078650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:24.078782 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:24.078843 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:24.079333 (+ 490us) server_negotiation.cc:299] Negotiation successful
0812 14:20:24.079464 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 14:20:24.079744 25667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.069784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:45843)
0812 14:20:24.069980 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:24.070015 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:24.070481 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:24.070677 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:24.070680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:24.071230 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:24.071477 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:24.071485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.072366 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.072368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:24.073431 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:24.073434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.073779 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.073780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:24.073924 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:24.074460 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:24.074469 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:24.076292 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:24.077922 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:24.077929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:24.077930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:24.078226 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:24.078424 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:24.078425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:24.078426 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:24.078586 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:24.078866 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:24.078873 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:24.079156 (+ 283us) client_negotiation.cc:769] Sending connection context
0812 14:20:24.079281 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 14:20:24.079485 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":116,"threads_started":1}
I20260812 14:20:24.080220 25665 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33235
I20260812 14:20:24.080329 25665 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:24.080497 25665 heartbeater.cc:511] Master 127.8.85.62:33235 requested a full tablet report, sending...
I20260812 14:20:24.080834 25137 ts_manager.cc:194] Registered new tserver with Master: 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981)
I20260812 14:20:24.081218 25137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:45843
I20260812 14:20:24.091327 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:20:24.094369 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.091766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36604 (local address 127.8.85.62:33235)
0812 14:20:24.091841 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:24.091843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:24.091953 (+ 110us) server_negotiation.cc:407] Connection header received
0812 14:20:24.092044 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:24.092045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:24.092099 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:24.092174 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:20:24.092313 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.092789 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.093583 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.093790 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.093867 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:24.094051 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:20:24.094153 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:20:24.094249 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:24.094272 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 14:20:24.094290 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 14:20:24.094693 25137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:20:24.096480 25137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:20:24.099035 25278 tablet_service.cc:1467] Processing CreateTablet for tablet 30ff3c8fde1a4130a96d8d0ac2b17c27 (DEFAULT_TABLE table=test-table [id=e2e42587df64457fb51cd0258577a0c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:20:24.099267 25278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30ff3c8fde1a4130a96d8d0ac2b17c27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:24.100857 25230 tablet_bootstrap.cc:492] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5: Bootstrap starting.
I20260812 14:20:24.101212 25230 tablet_bootstrap.cc:654] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:24.102129 25230 tablet_bootstrap.cc:492] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5: No bootstrap required, opened a new log
I20260812 14:20:24.102190 25230 ts_tablet_manager.cc:1397] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:24.102344 25230 raft_consensus.cc:348] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:24.102417 25230 raft_consensus.cc:374] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:24.102460 25230 raft_consensus.cc:729] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f27585b67c7146138f8ab173df86daf5, State: Initialized, Role: FOLLOWER
I20260812 14:20:24.102495 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.098468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:41981 (local address 127.0.0.1:56584)
0812 14:20:24.098607 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:24.098619 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:24.098662 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:24.099151 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:24.099151 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:24.099152 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:24.099182 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:24.099183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.100375 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.100376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:24.101415 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:24.101419 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.102212 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.102213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:24.102283 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:24.102306 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 14:20:24.102363 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 14:20:24.102410 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260812 14:20:24.102519 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.098778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56584 (local address 127.8.85.3:41981)
0812 14:20:24.098962 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:24.098967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:24.098977 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:20:24.099015 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:24.099017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:24.099031 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:24.099146 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:24.099225 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.100261 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.101523 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.102125 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.102206 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:24.102277 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:24.102377 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 14:20:24.102429 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":102,"threads_started":1}
I20260812 14:20:24.102507 25230 consensus_queue.cc:260] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:24.102612 25230 ts_tablet_manager.cc:1428] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:24.103230 25588 tablet_service.cc:1467] Processing CreateTablet for tablet 30ff3c8fde1a4130a96d8d0ac2b17c27 (DEFAULT_TABLE table=test-table [id=e2e42587df64457fb51cd0258577a0c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:20:24.103657 25588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30ff3c8fde1a4130a96d8d0ac2b17c27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:24.103749 25672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.098495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:35887 (local address 127.0.0.1:42668)
0812 14:20:24.098656 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:24.098668 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:24.098712 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:24.098997 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:24.098998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:24.099000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:24.099044 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:24.099046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.100112 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.100113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:24.102396 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:24.102398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.103509 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.103510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:24.103577 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:24.103581 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:20:24.103636 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 14:20:24.103681 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":107,"threads_started":1}
I20260812 14:20:24.103813 25673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.098595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42668 (local address 127.8.85.2:35887)
0812 14:20:24.098808 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:24.098815 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:24.098826 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:20:24.098871 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:24.098873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:24.098900 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:24.098982 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:24.099106 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.099997 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.102487 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.103306 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.103357 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:24.103441 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:24.103654 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 14:20:24.103696 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":106,"threads_started":1}
I20260812 14:20:24.104512 25436 tablet_service.cc:1467] Processing CreateTablet for tablet 30ff3c8fde1a4130a96d8d0ac2b17c27 (DEFAULT_TABLE table=test-table [id=e2e42587df64457fb51cd0258577a0c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:20:24.104893 25436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30ff3c8fde1a4130a96d8d0ac2b17c27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:24.107450 25676 tablet_bootstrap.cc:492] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4: Bootstrap starting.
I20260812 14:20:24.108172 25676 tablet_bootstrap.cc:654] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:24.108430 25677 tablet_bootstrap.cc:492] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70: Bootstrap starting.
I20260812 14:20:24.108832 25676 log.cc:824] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:24.109167 25677 tablet_bootstrap.cc:654] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:24.109628 25676 tablet_bootstrap.cc:492] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4: No bootstrap required, opened a new log
I20260812 14:20:24.109648 25677 log.cc:824] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:24.109721 25676 ts_tablet_manager.cc:1397] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:20:24.110349 25677 tablet_bootstrap.cc:492] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70: No bootstrap required, opened a new log
I20260812 14:20:24.110404 25677 ts_tablet_manager.cc:1397] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:20:24.111229 25676 raft_consensus.cc:348] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:24.111546 25676 raft_consensus.cc:374] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:24.111631 25676 raft_consensus.cc:729] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81e60aaa3af74dbbb65d148f95028ab4, State: Initialized, Role: FOLLOWER
I20260812 14:20:24.111763 25676 consensus_queue.cc:260] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:24.112064 25676 ts_tablet_manager.cc:1428] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:20:24.112139 25665 heartbeater.cc:503] Master 127.8.85.62:33235 was elected leader, sending a full tablet report...
I20260812 14:20:24.112197 25677 raft_consensus.cc:348] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:24.112392 25677 raft_consensus.cc:374] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:24.112433 25677 raft_consensus.cc:729] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f677095999642a1a2d10ba4066bde70, State: Initialized, Role: FOLLOWER
I20260812 14:20:24.112550 25677 consensus_queue.cc:260] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:24.112855 25677 ts_tablet_manager.cc:1428] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:20:24.112923 25512 heartbeater.cc:503] Master 127.8.85.62:33235 was elected leader, sending a full tablet report...
W20260812 14:20:24.192585 25514 tablet.cc:2369] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:20:24.262954 25681 raft_consensus.cc:482] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:20:24.263092 25681 raft_consensus.cc:504] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:24.263744 25681 leader_election.cc:288] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887), f27585b67c7146138f8ab173df86daf5 (127.8.85.1:34593)
I20260812 14:20:24.266935 25667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.263949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:35887 (local address 127.8.85.3:39085)
0812 14:20:24.264010 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:24.264022 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:24.264070 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:24.264291 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:24.264292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:24.264294 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:24.264387 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:24.264389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.265158 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.265159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:24.266007 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:24.266010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.266650 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.266651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:24.266738 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:24.266743 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:20:24.266808 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 14:20:24.266856 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 14:20:24.266947 25673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.264032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:39085 (local address 127.8.85.2:35887)
0812 14:20:24.264094 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:24.264096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:24.264127 (+ 31us) server_negotiation.cc:407] Connection header received
0812 14:20:24.264163 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:24.264164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:24.264180 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:24.264236 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:24.264500 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.265063 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.266112 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.266552 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.266607 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:24.266615 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:24.266827 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 14:20:24.266858 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 14:20:24.267215 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.264017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:42901 (local address 127.8.85.1:34593)
0812 14:20:24.264107 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:24.264109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:24.264267 (+ 158us) server_negotiation.cc:407] Connection header received
0812 14:20:24.264300 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:24.264301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:24.264333 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:24.264426 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:24.264676 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.265287 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.266317 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.266829 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.266904 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:24.266992 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:24.267103 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 14:20:24.267152 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 14:20:24.267467 25683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.263961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:34593 (local address 127.8.85.3:42901)
0812 14:20:24.264147 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:24.264156 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:24.264220 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:24.264501 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:24.264502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:24.264504 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:24.264582 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:24.264584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.265381 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.265382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:24.266211 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:24.266214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.266914 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.266915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:24.266995 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:24.267000 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:20:24.267304 (+ 304us) client_negotiation.cc:240] Negotiation successful
0812 14:20:24.267356 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":105,"threads_started":1}
I20260812 14:20:24.267382 25457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ff3c8fde1a4130a96d8d0ac2b17c27" candidate_uuid: "81e60aaa3af74dbbb65d148f95028ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f677095999642a1a2d10ba4066bde70" is_pre_election: true
I20260812 14:20:24.267629 25457 raft_consensus.cc:2454] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81e60aaa3af74dbbb65d148f95028ab4 in term 0.
I20260812 14:20:24.267802 25302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ff3c8fde1a4130a96d8d0ac2b17c27" candidate_uuid: "81e60aaa3af74dbbb65d148f95028ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f27585b67c7146138f8ab173df86daf5" is_pre_election: true
I20260812 14:20:24.267875 25539 leader_election.cc:302] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81e60aaa3af74dbbb65d148f95028ab4, 8f677095999642a1a2d10ba4066bde70; no voters:
I20260812 14:20:24.267974 25302 raft_consensus.cc:2454] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81e60aaa3af74dbbb65d148f95028ab4 in term 0.
I20260812 14:20:24.268023 25681 raft_consensus.cc:2781] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:20:24.268077 25681 raft_consensus.cc:482] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:20:24.268113 25681 raft_consensus.cc:3037] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:24.268878 25681 raft_consensus.cc:504] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:24.269040 25681 leader_election.cc:288] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [CANDIDATE]: Term 1 election: Requested vote from peers 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887), f27585b67c7146138f8ab173df86daf5 (127.8.85.1:34593)
I20260812 14:20:24.269220 25457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ff3c8fde1a4130a96d8d0ac2b17c27" candidate_uuid: "81e60aaa3af74dbbb65d148f95028ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f677095999642a1a2d10ba4066bde70"
I20260812 14:20:24.269234 25302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ff3c8fde1a4130a96d8d0ac2b17c27" candidate_uuid: "81e60aaa3af74dbbb65d148f95028ab4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f27585b67c7146138f8ab173df86daf5"
I20260812 14:20:24.269315 25302 raft_consensus.cc:3037] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:24.269328 25457 raft_consensus.cc:3037] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:24.270035 25302 raft_consensus.cc:2454] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81e60aaa3af74dbbb65d148f95028ab4 in term 1.
I20260812 14:20:24.270138 25457 raft_consensus.cc:2454] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81e60aaa3af74dbbb65d148f95028ab4 in term 1.
I20260812 14:20:24.270234 25538 leader_election.cc:302] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81e60aaa3af74dbbb65d148f95028ab4, f27585b67c7146138f8ab173df86daf5; no voters:
I20260812 14:20:24.270351 25681 raft_consensus.cc:2781] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:20:24.270526 25681 raft_consensus.cc:686] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [term 1 LEADER]: Becoming Leader. State: Replica: 81e60aaa3af74dbbb65d148f95028ab4, State: Running, Role: LEADER
I20260812 14:20:24.270635 25681 consensus_queue.cc:237] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:24.271587 25137 catalog_manager.cc:5495] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3). New cstate: current_term: 1 leader_uuid: "81e60aaa3af74dbbb65d148f95028ab4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } health_report { overall_health: UNKNOWN } } }
I20260812 14:20:24.285486 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.283022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56586 (local address 127.8.85.3:41981)
0812 14:20:24.283088 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:24.283090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:24.283102 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:20:24.283131 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:24.283131 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:24.283148 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:24.283231 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:20:24.283376 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.283889 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.284641 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:24.284860 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:24.284940 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:24.285113 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:20:24.285216 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:20:24.285304 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:24.285350 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 14:20:24.285372 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:20:24.287283 25457 raft_consensus.cc:1261] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70 [term 1 FOLLOWER]: Refusing update from remote peer 81e60aaa3af74dbbb65d148f95028ab4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:20:24.287283 25302 raft_consensus.cc:1261] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Refusing update from remote peer 81e60aaa3af74dbbb65d148f95028ab4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:20:24.287712 25681 consensus_queue.cc:1035] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:20:24.287889 25686 consensus_queue.cc:1035] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:20:24.291136 25691 mvcc.cc:205] Tried to move back new op lower bound from 7317685961877929984 to 7317685961813405696. Current Snapshot: MvccSnapshot[applied={T|T < 7317685961877929984}]
I20260812 14:20:24.292912 25136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:20:24.293020 25136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:20:24.293807 25136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: Sending DeleteTablet for 3 replicas of tablet 30ff3c8fde1a4130a96d8d0ac2b17c27
I20260812 14:20:24.294126 25588 tablet_service.cc:1514] Processing DeleteTablet for tablet 30ff3c8fde1a4130a96d8d0ac2b17c27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:20:24 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:56584
I20260812 14:20:24.294186 25278 tablet_service.cc:1514] Processing DeleteTablet for tablet 30ff3c8fde1a4130a96d8d0ac2b17c27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:20:24 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:42012
I20260812 14:20:24.294235 25436 tablet_service.cc:1514] Processing DeleteTablet for tablet 30ff3c8fde1a4130a96d8d0ac2b17c27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:20:24 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:42668
I20260812 14:20:24.294350 25702 tablet_replica.cc:330] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4: stopping tablet replica
I20260812 14:20:24.294437 25366 tablet_replica.cc:330] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5: stopping tablet replica
I20260812 14:20:24.294512 25366 raft_consensus.cc:2229] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:20:24.294512 25702 raft_consensus.cc:2229] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:20:24.294564 25366 raft_consensus.cc:2258] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:20:24.294613 25702 raft_consensus.cc:2258] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:20:24.294837 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 25183
I20260812 14:20:24.294852 25704 tablet_replica.cc:330] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70: stopping tablet replica
I20260812 14:20:24.300697 25704 raft_consensus.cc:2229] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:20:24.300848 25704 raft_consensus.cc:2258] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:20:24.301498 25704 ts_tablet_manager.cc:1905] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:20:24.303484 25704 ts_tablet_manager.cc:1918] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:20:24.303566 25704 log.cc:1196] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/30ff3c8fde1a4130a96d8d0ac2b17c27
I20260812 14:20:24.303886 25704 ts_tablet_manager.cc:1939] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 8f677095999642a1a2d10ba4066bde70: Deleting consensus metadata
I20260812 14:20:24.304409 25124 catalog_manager.cc:4841] TS 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887): tablet 30ff3c8fde1a4130a96d8d0ac2b17c27 (table test-table [id=e2e42587df64457fb51cd0258577a0c3]) successfully deleted
I20260812 14:20:24.310992 25702 ts_tablet_manager.cc:1905] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:20:24.313073 25702 ts_tablet_manager.cc:1918] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:20:24.313164 25702 log.cc:1196] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/30ff3c8fde1a4130a96d8d0ac2b17c27
I20260812 14:20:24.313511 25702 ts_tablet_manager.cc:1939] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P 81e60aaa3af74dbbb65d148f95028ab4: Deleting consensus metadata
I20260812 14:20:24.314030 25123 catalog_manager.cc:4841] TS 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981): tablet 30ff3c8fde1a4130a96d8d0ac2b17c27 (table test-table [id=e2e42587df64457fb51cd0258577a0c3]) successfully deleted
W20260812 14:20:24.315686 25123 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:34593 (error 108)
I20260812 14:20:24.316126 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 25361
I20260812 14:20:24.316314 25672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.316145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:34593 (local address 127.0.0.1:56700)
0812 14:20:24.316222 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:24.316265 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:34593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 14:20:24.317245 25123 catalog_manager.cc:4584] TS f27585b67c7146138f8ab173df86daf5 (127.8.85.1:34593): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 30ff3c8fde1a4130a96d8d0ac2b17c27: Network error: Client connection negotiation failed: client connection to 127.8.85.1:34593: connect: Connection refused (error 111)
I20260812 14:20:24.324472 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 25517
I20260812 14:20:24.335516 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 25102
I20260812 14:20:24.349391 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:33235
--webserver_interface=127.8.85.62
--webserver_port=43817
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:33235
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:24.446199 25706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:24.446437 25706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:24.446463 25706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:24.447705 25706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:20:24.447745 25706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:24.447760 25706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:24.447774 25706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:20:24.447789 25706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:20:24.449419 25706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:33235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:33235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=43817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:24.449636 25706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:24.449831 25706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:24.455085 25715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:24.455117 25713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:24.455147 25706 server_base.cc:1034] running on GCE node
W20260812 14:20:24.455090 25712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:24.455648 25706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:24.455991 25706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:24.457175 25706 hybrid_clock.cc:648] HybridClock initialized: now 1786544424457152 us; error 32 us; skew 500 ppm
Aug 12 14:20:24 dist-test-slave-5k54 krb5kdc[24475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544424, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:24.459913 25706 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:20:24.460414 25706 webserver.cc:460] Webserver started at http://127.8.85.62:43817/ using document root <none> and password file <none>
I20260812 14:20:24.460668 25706 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:24.460741 25706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:24.462208 25706 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 14:20:24.462929 25722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:24.463100 25706 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:20:24.463209 25706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "b17f8b1fb57f44e7a38d482552e2ce3e"
format_stamp: "Formatted at 2026-08-12 14:20:22 on dist-test-slave-5k54"
server_key: "3f65701e554262378f45578b70895322"
server_key_iv: "2df3d3c7d27399fc5ae93576f3a4ed45"
server_key_version: "encryptionkey@0"
I20260812 14:20:24.463632 25706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:24.490160 25706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:24.492560 25706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:24.492725 25706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:24.497630 25706 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:33235
I20260812 14:20:24.497668 25780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:33235 every 8 connection(s)
I20260812 14:20:24.498075 25706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:20:24.499670 25781 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:20:24.500273 25781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: Bootstrap starting.
I20260812 14:20:24.506271 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 25706
I20260812 14:20:24.506633 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:34593
--local_ip_for_outbound_sockets=127.8.85.1
--tserver_master_addrs=127.8.85.62:33235
--webserver_port=43945
--webserver_interface=127.8.85.1
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:20:24.509490 25781 log.cc:824] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:24.512317 25781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:20:24.512670 25781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e: Bootstrap complete.
I20260812 14:20:24.515035 25781 raft_consensus.cc:348] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } }
I20260812 14:20:24.515291 25781 raft_consensus.cc:729] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17f8b1fb57f44e7a38d482552e2ce3e, State: Initialized, Role: FOLLOWER
I20260812 14:20:24.515497 25781 consensus_queue.cc:260] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } }
I20260812 14:20:24.515580 25781 raft_consensus.cc:388] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:20:24.515635 25781 raft_consensus.cc:482] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:20:24.515683 25781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 2 FOLLOWER]: Advancing to term 3
I20260812 14:20:24.516687 25781 raft_consensus.cc:504] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } }
I20260812 14:20:24.516809 25781 leader_election.cc:302] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b17f8b1fb57f44e7a38d482552e2ce3e; no voters:
I20260812 14:20:24.517020 25781 leader_election.cc:288] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 14:20:24.517056 25786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 3 FOLLOWER]: Leader election won for term 3
I20260812 14:20:24.517164 25786 raft_consensus.cc:686] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [term 3 LEADER]: Becoming Leader. State: Replica: b17f8b1fb57f44e7a38d482552e2ce3e, State: Running, Role: LEADER
I20260812 14:20:24.517297 25781 sys_catalog.cc:564] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:20:24.517283 25786 consensus_queue.cc:237] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } }
I20260812 14:20:24.517604 25786 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b17f8b1fb57f44e7a38d482552e2ce3e. Latest consensus state: current_term: 3 leader_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } } }
I20260812 14:20:24.517699 25786 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [sys.catalog]: This master's current role is: LEADER
I20260812 14:20:24.518038 25791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:20:24.518886 25791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:20:24.519019 25791 catalog_manager.cc:1259] Loaded cluster ID: 060c4c96b5384723b62291ae4de3610e
I20260812 14:20:24.519063 25791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:20:24.519294 25787 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17f8b1fb57f44e7a38d482552e2ce3e" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33235 } } }
F20260812 14:20:24.519407 25791 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
I20260812 14:20:24.519582 25787 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17f8b1fb57f44e7a38d482552e2ce3e [sys.catalog]: This master's current role is: LEADER
@ 0x7fb9513699ad google::LogMessage::Fail() at ??:0
@ 0x7fb95136c88a google::LogMessage::SendToLog() at ??:0
@ 0x7fb9513694de google::LogMessage::Flush() at ??:0
@ 0x7fb95136aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb95928a0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fb9592938b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb9532370d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb95322c954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb9540906db start_thread at ??:0
@ 0x7fb94fe6761f clone at ??:0
W20260812 14:20:24.603626 25784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:24.603842 25784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:24.603881 25784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:24.605008 25784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:24.605072 25784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:24.605115 25784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:20:24.606560 25784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:34593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=43945
--tserver_master_addrs=127.8.85.62:33235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:24.606827 25784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:24.607084 25784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:24.612135 25827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:24.612147 25825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:24.612114 25830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:24.612630 25784 server_base.cc:1034] running on GCE node
I20260812 14:20:24.612833 25784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:24.613090 25784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:24.614266 25784 hybrid_clock.cc:648] HybridClock initialized: now 1786544424614252 us; error 27 us; skew 500 ppm
Aug 12 14:20:24 dist-test-slave-5k54 krb5kdc[24475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544424, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:24.617316 25784 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:20:24.617890 25784 webserver.cc:460] Webserver started at http://127.8.85.1:43945/ using document root <none> and password file <none>
I20260812 14:20:24.618175 25784 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:24.618250 25784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:24.620074 25784 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:20:24.620913 25837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:24.621112 25784 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:24.621214 25784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "f27585b67c7146138f8ab173df86daf5"
format_stamp: "Formatted at 2026-08-12 14:20:23 on dist-test-slave-5k54"
server_key: "4176f2ee2f5fd5a9769ebca34c8094f6"
server_key_iv: "de0d291af584064b45c375bf2327e9d2"
server_key_version: "encryptionkey@0"
I20260812 14:20:24.621634 25784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:24.644310 25784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:24.646909 25784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:24.647064 25784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:24.647326 25784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:24.647892 25846 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 14:20:24.649434 25784 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 14:20:24.649497 25784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 14:20:24.649533 25784 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 14:20:24.650478 25784 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 14:20:24.650528 25784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 14:20:24.650602 25846 tablet_bootstrap.cc:492] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5: Bootstrap starting.
I20260812 14:20:24.657812 25784 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:34593
I20260812 14:20:24.658198 25784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:20:24.659534 25966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:34593 every 8 connection(s)
I20260812 14:20:24.661229 25846 log.cc:824] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:24.663396 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 25784
I20260812 14:20:24.663704 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:35887
--local_ip_for_outbound_sockets=127.8.85.2
--tserver_master_addrs=127.8.85.62:33235
--webserver_port=43185
--webserver_interface=127.8.85.2
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:20:24.663939 25846 tablet_bootstrap.cc:492] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:20:24.664263 25846 tablet_bootstrap.cc:492] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5: Bootstrap complete.
I20260812 14:20:24.665442 25846 ts_tablet_manager.cc:1397] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260812 14:20:24.666895 25846 raft_consensus.cc:348] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:24.667410 25846 raft_consensus.cc:729] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f27585b67c7146138f8ab173df86daf5, State: Initialized, Role: FOLLOWER
I20260812 14:20:24.667605 25846 consensus_queue.cc:260] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:24.667943 25846 ts_tablet_manager.cc:1428] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5: Time spent starting tablet: real 0.002s user 0.001s sys 0.003s
W20260812 14:20:24.748921 25972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:24.749162 25972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:24.749204 25972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:24.750375 25972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:24.750450 25972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:24.750497 25972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:20:24.752051 25972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:35887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=43185
--tserver_master_addrs=127.8.85.62:33235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:24.752321 25972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:24.752573 25972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:20:24.757853 25972 server_base.cc:1034] running on GCE node
W20260812 14:20:24.757822 25983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:24.757824 25980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:24.758056 25981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:24.758283 25972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:24.758555 25972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:24.759742 25972 hybrid_clock.cc:648] HybridClock initialized: now 1786544424759714 us; error 40 us; skew 500 ppm
Aug 12 14:20:24 dist-test-slave-5k54 krb5kdc[24475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544424, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:24.763149 25972 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:20:24.763720 25972 webserver.cc:460] Webserver started at http://127.8.85.2:43185/ using document root <none> and password file <none>
I20260812 14:20:24.763975 25972 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:24.764050 25972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:24.765539 25972 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:20:24.766220 25991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:24.766417 25972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:24.766526 25972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "8f677095999642a1a2d10ba4066bde70"
format_stamp: "Formatted at 2026-08-12 14:20:23 on dist-test-slave-5k54"
server_key: "8bf7e1298d8ceceda27f3951d9074905"
server_key_iv: "2afc0a3f646b015b9bd763d17981f625"
server_key_version: "encryptionkey@0"
I20260812 14:20:24.766927 25972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260812 14:20:24.767549 25970 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:20:24.658920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:43529)
0812 14:20:24.659093 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:24.659137 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:24.659731 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:24.767299 (+107568us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
W20260812 14:20:24.767764 25968 heartbeater.cc:644] Failed to heartbeat to 127.8.85.62:33235 (0 consecutive failures): Network error: Failed to ping master at 127.8.85.62:33235: Client connection negotiation failed: client connection to 127.8.85.62:33235: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 14:20:24.768149 25970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.768024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:60813)
0812 14:20:24.768086 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:24.768128 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 14:20:24.768493 25970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.768395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:36813)
0812 14:20:24.768441 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:24.768467 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 14:20:24.768603 25968 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 14:20:24.788460 25972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:24.791002 25972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:24.791184 25972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:24.791509 25972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:24.791844 25972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:24.791900 25972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:24.791955 25972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:24.791986 25972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:24.798117 25972 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:35887
I20260812 14:20:24.798166 26120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:35887 every 8 connection(s)
I20260812 14:20:24.798527 25972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:20:24.799626 26123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.799160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:33391)
0812 14:20:24.799320 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:24.799467 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
W20260812 14:20:24.800055 26121 heartbeater.cc:644] Failed to heartbeat to 127.8.85.62:33235 (0 consecutive failures): Network error: Failed to ping master at 127.8.85.62:33235: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
I20260812 14:20:24.800350 26123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.800257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:49383)
0812 14:20:24.800303 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:24.800333 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 14:20:24.800657 26123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.800580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:37427)
0812 14:20:24.800624 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:24.800643 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 14:20:24.800776 26121 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 14:20:24.801430 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 25972
I20260812 14:20:24.801640 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:41981
--local_ip_for_outbound_sockets=127.8.85.3
--tserver_master_addrs=127.8.85.62:33235
--webserver_port=45947
--webserver_interface=127.8.85.3
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:24.889827 26124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:24.890067 26124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:24.890111 26124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:24.891289 26124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:24.891357 26124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:24.891481 26124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:20:24.893003 26124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:41981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=45947
--tserver_master_addrs=127.8.85.62:33235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:24.893270 26124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:24.893518 26124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:24.898571 26134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:24.898576 26131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:24.898782 26124 server_base.cc:1034] running on GCE node
W20260812 14:20:24.898659 26132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:24.899073 26124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:24.899325 26124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:24.900499 26124 hybrid_clock.cc:648] HybridClock initialized: now 1786544424900497 us; error 42 us; skew 500 ppm
Aug 12 14:20:24 dist-test-slave-5k54 krb5kdc[24475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544424, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:24.903059 26124 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:20:24.903623 26124 webserver.cc:460] Webserver started at http://127.8.85.3:45947/ using document root <none> and password file <none>
I20260812 14:20:24.903884 26124 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:24.903954 26124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:24.905308 26124 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:24.905928 26142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:24.906100 26124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:20:24.906193 26124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "81e60aaa3af74dbbb65d148f95028ab4"
format_stamp: "Formatted at 2026-08-12 14:20:23 on dist-test-slave-5k54"
server_key: "98b452e3c86d3e05156cf3264c24d663"
server_key_iv: "448f218adc57901e18183c5e95183308"
server_key_version: "encryptionkey@0"
I20260812 14:20:24.906570 26124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:24.920357 26124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:24.922462 26124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:24.922634 26124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:24.922883 26124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:24.923202 26124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:24.923257 26124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:24.923300 26124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:24.923344 26124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:24.929840 26124 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:41981
I20260812 14:20:24.929896 26270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:41981 every 8 connection(s)
I20260812 14:20:24.930251 26124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:20:24.931321 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.930950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:33019)
0812 14:20:24.931110 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:24.931183 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
W20260812 14:20:24.931766 26272 heartbeater.cc:644] Failed to heartbeat to 127.8.85.62:33235 (0 consecutive failures): Network error: Failed to ping master at 127.8.85.62:33235: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
I20260812 14:20:24.932106 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.932013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:47579)
0812 14:20:24.932059 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:24.932088 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 14:20:24.932391 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:24.932317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:52771)
0812 14:20:24.932355 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:24.932377 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 14:20:24.932539 26272 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 14:20:24.937934 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 26124
I20260812 14:20:25.769192 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:25.768935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:55325)
0812 14:20:25.769083 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:25.769127 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":64,"threads_started":1}
I20260812 14:20:25.801296 26277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:25.801090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:52459)
0812 14:20:25.801207 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:25.801244 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260812 14:20:25.933152 26279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:25.932926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:39935)
0812 14:20:25.933068 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:25.933103 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":60,"threads_started":1}
I20260812 14:20:26.245090 26280 raft_consensus.cc:482] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:20:26.245261 26280 raft_consensus.cc:504] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:26.245644 26280 leader_election.cc:288] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981), 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887)
Aug 12 14:20:26 dist-test-slave-5k54 krb5kdc[24475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544424, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.2@KRBTEST.COM
Aug 12 14:20:26 dist-test-slave-5k54 krb5kdc[24475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544424, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.3@KRBTEST.COM
I20260812 14:20:26.256373 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:26.245806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:41981 (local address 127.8.85.1:34501)
0812 14:20:26.245942 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:26.245960 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:26.246012 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:26.246884 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:26.246886 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:26.247310 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:26.247560 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:26.247566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:26.248638 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:26.248640 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:26.249434 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:26.249436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:26.250048 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:26.250049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:26.250172 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:26.250866 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:26.250875 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:26.252887 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:26.254831 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:26.254832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:26.254832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:26.255093 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:26.255386 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:26.255387 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:26.255387 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:26.255491 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:26.255899 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:26.255900 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:26.255983 (+ 83us) client_negotiation.cc:769] Sending connection context
0812 14:20:26.256055 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:20:26.256268 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":13}
I20260812 14:20:26.256390 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:26.245957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:45865 (local address 127.8.85.2:35887)
0812 14:20:26.246109 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:26.246123 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:26.246182 (+ 59us) server_negotiation.cc:407] Connection header received
0812 14:20:26.246653 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:26.246655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:26.246696 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:26.246860 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:26.247692 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:26.248540 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:26.249551 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:26.249940 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:26.250058 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:26.252611 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:26.252615 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:26.252616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:26.252625 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:26.254509 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:26.255045 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:26.255048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:26.255048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:26.255167 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:26.255518 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:26.255519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:26.255519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:26.255775 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:26.255859 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:26.256172 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 14:20:26.256292 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260812 14:20:26.256445 26283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:26.245908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:34501 (local address 127.8.85.3:41981)
0812 14:20:26.246107 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:26.246122 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:26.246158 (+ 36us) server_negotiation.cc:407] Connection header received
0812 14:20:26.246655 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:26.246657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:26.246696 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:26.246841 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:26.247693 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:26.248534 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:26.249530 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:26.249938 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:26.250058 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:26.252994 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:26.252998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:26.252999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:26.253011 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:26.254747 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:26.255178 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:26.255181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:26.255181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:26.255315 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:26.255567 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:26.255568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:26.255568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:26.255786 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:26.255891 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:26.256157 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 14:20:26.256327 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":87,"threads_started":1}
I20260812 14:20:26.256489 26282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:26.245808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:35887 (local address 127.8.85.1:45865)
0812 14:20:26.246112 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:26.246123 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:26.246197 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:26.246884 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:26.246887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:26.247311 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:26.247560 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:26.247566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:26.248670 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:26.248671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:26.249436 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:26.249438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:26.250045 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:26.250046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:26.250169 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:26.250931 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:26.250935 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:26.252510 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:26.254606 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:26.254614 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:26.254616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:26.254961 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:26.255237 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:26.255238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:26.255238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:26.255400 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:26.255853 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:26.255861 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:26.255992 (+ 131us) client_negotiation.cc:769] Sending connection context
0812 14:20:26.256080 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 14:20:26.256348 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":213,"thread_start_us":48,"threads_started":1}
W20260812 14:20:26.257777 25841 leader_election.cc:341] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887): Not found: Tablet not found: 30ff3c8fde1a4130a96d8d0ac2b17c27
W20260812 14:20:26.257874 25840 leader_election.cc:341] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981): Not found: Tablet not found: 30ff3c8fde1a4130a96d8d0ac2b17c27
I20260812 14:20:26.257899 25840 leader_election.cc:302] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f27585b67c7146138f8ab173df86daf5; no voters: 81e60aaa3af74dbbb65d148f95028ab4, 8f677095999642a1a2d10ba4066bde70
I20260812 14:20:26.258009 26280 raft_consensus.cc:2726] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:20:26.769887 26285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:26.769682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:48643)
0812 14:20:26.769803 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:26.769838 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260812 14:20:26.801997 26286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:26.801792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:42287)
0812 14:20:26.801924 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:26.801957 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260812 14:20:26.933858 26287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:26.933586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:47035)
0812 14:20:26.933721 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:26.933762 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260812 14:20:27.770572 26288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:27.770347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:48255)
0812 14:20:27.770487 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:27.770523 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260812 14:20:27.802743 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:27.802511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:38279)
0812 14:20:27.802653 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:27.802697 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260812 14:20:27.934571 26290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:27.934346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:60791)
0812 14:20:27.934489 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:27.934529 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":68,"threads_started":1}
I20260812 14:20:28.010766 26291 raft_consensus.cc:482] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:20:28.010869 26291 raft_consensus.cc:504] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:28.011086 26291 leader_election.cc:288] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981), 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887)
W20260812 14:20:28.011670 25841 leader_election.cc:341] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887): Not found: Tablet not found: 30ff3c8fde1a4130a96d8d0ac2b17c27
W20260812 14:20:28.011806 25840 leader_election.cc:341] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981): Not found: Tablet not found: 30ff3c8fde1a4130a96d8d0ac2b17c27
I20260812 14:20:28.011889 25840 leader_election.cc:302] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f27585b67c7146138f8ab173df86daf5; no voters: 81e60aaa3af74dbbb65d148f95028ab4, 8f677095999642a1a2d10ba4066bde70
I20260812 14:20:28.011965 26291 raft_consensus.cc:2726] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:20:28.771311 26292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:28.771000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:54673)
0812 14:20:28.771162 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:28.771198 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260812 14:20:28.803565 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:28.803294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:42193)
0812 14:20:28.803476 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:28.803518 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":91,"threads_started":1}
I20260812 14:20:28.935290 26294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:28.935038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:57443)
0812 14:20:28.935168 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:28.935207 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260812 14:20:29.772086 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:29.771846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:47489)
0812 14:20:29.771988 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:29.772025 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260812 14:20:29.804265 26296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:29.804045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:59653)
0812 14:20:29.804184 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:29.804222 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260812 14:20:29.935949 26297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:29.935746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:50103)
0812 14:20:29.935868 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:29.935906 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":59,"threads_started":1}
I20260812 14:20:30.540230 26298 raft_consensus.cc:482] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:20:30.540360 26298 raft_consensus.cc:504] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:30.540582 26298 leader_election.cc:288] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981), 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887)
W20260812 14:20:30.540944 25840 leader_election.cc:341] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981): Not found: Tablet not found: 30ff3c8fde1a4130a96d8d0ac2b17c27
W20260812 14:20:30.541045 25841 leader_election.cc:341] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887): Not found: Tablet not found: 30ff3c8fde1a4130a96d8d0ac2b17c27
I20260812 14:20:30.541136 25841 leader_election.cc:302] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f27585b67c7146138f8ab173df86daf5; no voters: 81e60aaa3af74dbbb65d148f95028ab4, 8f677095999642a1a2d10ba4066bde70
I20260812 14:20:30.541209 26298 raft_consensus.cc:2726] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:20:30.772764 26300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:30.772546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:34973)
0812 14:20:30.772676 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:30.772718 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260812 14:20:30.804976 26301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:30.804725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:49107)
0812 14:20:30.804890 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:30.804932 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260812 14:20:30.936542 26302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:30.936351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:58173)
0812 14:20:30.936463 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:30.936499 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
I20260812 14:20:31.773414 26304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:31.773202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:41673)
0812 14:20:31.773325 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:31.773365 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260812 14:20:31.805665 26305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:31.805433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:42797)
0812 14:20:31.805586 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:31.805617 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260812 14:20:31.937208 26307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:31.937001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:35371)
0812 14:20:31.937129 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:31.937168 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260812 14:20:32.774112 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:32.773887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:39765)
0812 14:20:32.774023 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:32.774063 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260812 14:20:32.806493 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:32.806226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:45477)
0812 14:20:32.806397 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:32.806441 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260812 14:20:32.938007 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:32.937780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:53361)
0812 14:20:32.937928 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:32.937964 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260812 14:20:33.774883 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:33.774635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:32961)
0812 14:20:33.774789 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:33.774834 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":63,"threads_started":1}
I20260812 14:20:33.807207 26314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:33.806988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:40573)
0812 14:20:33.807119 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:33.807159 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260812 14:20:33.938733 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:33.938511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:58289)
0812 14:20:33.938642 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:33.938685 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260812 14:20:34.775665 26316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:34.775398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:57211)
0812 14:20:34.775580 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:34.775618 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":122,"threads_started":1}
I20260812 14:20:34.807919 26317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:34.807699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:58809)
0812 14:20:34.807827 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:34.807868 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260812 14:20:34.939548 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:34.939318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:35185)
0812 14:20:34.939471 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:34.939500 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
I20260812 14:20:35.196201 26319 raft_consensus.cc:482] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:20:35.196302 26319 raft_consensus.cc:504] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:35.196467 26319 leader_election.cc:288] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981), 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887)
W20260812 14:20:35.196903 25840 leader_election.cc:341] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981): Not found: Tablet not found: 30ff3c8fde1a4130a96d8d0ac2b17c27
W20260812 14:20:35.197005 25841 leader_election.cc:341] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887): Not found: Tablet not found: 30ff3c8fde1a4130a96d8d0ac2b17c27
I20260812 14:20:35.197082 25841 leader_election.cc:302] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f27585b67c7146138f8ab173df86daf5; no voters: 81e60aaa3af74dbbb65d148f95028ab4, 8f677095999642a1a2d10ba4066bde70
I20260812 14:20:35.197180 26319 raft_consensus.cc:2726] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:20:35.776407 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:35.776165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:55309)
0812 14:20:35.776311 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:35.776358 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260812 14:20:35.808682 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:35.808421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:57785)
0812 14:20:35.808597 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:35.808638 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":114,"threads_started":1}
I20260812 14:20:35.940318 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:35.940039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:50061)
0812 14:20:35.940218 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:35.940267 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":85,"threads_started":1}
I20260812 14:20:36.777283 26323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:36.777023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:37621)
0812 14:20:36.777188 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:36.777235 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":70,"threads_started":1}
I20260812 14:20:36.809482 26325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:36.809249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:39625)
0812 14:20:36.809397 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:36.809436 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260812 14:20:36.941123 26326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:36.940874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:34061)
0812 14:20:36.941030 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:36.941080 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260812 14:20:37.778082 26327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:37.777838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:55431)
0812 14:20:37.777998 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:37.778034 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":79,"threads_started":1}
I20260812 14:20:37.810235 26328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:37.810009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:50299)
0812 14:20:37.810145 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:37.810185 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
I20260812 14:20:37.941883 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:37.941648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:41787)
0812 14:20:37.941796 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:37.941838 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260812 14:20:38.778916 26330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:38.778682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:49575)
0812 14:20:38.778829 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:38.778870 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":61,"threads_started":1}
I20260812 14:20:38.810925 26331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:38.810710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:34985)
0812 14:20:38.810846 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:38.810880 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260812 14:20:38.942514 26332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:38.942305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:35987)
0812 14:20:38.942439 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:38.942469 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
I20260812 14:20:39.064888 26333 raft_consensus.cc:482] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:20:39.064990 26333 raft_consensus.cc:504] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81e60aaa3af74dbbb65d148f95028ab4" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 41981 } } peers { permanent_uuid: "8f677095999642a1a2d10ba4066bde70" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 35887 } } peers { permanent_uuid: "f27585b67c7146138f8ab173df86daf5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 34593 } }
I20260812 14:20:39.065205 26333 leader_election.cc:288] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981), 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887)
W20260812 14:20:39.065701 25841 leader_election.cc:341] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f677095999642a1a2d10ba4066bde70 (127.8.85.2:35887): Not found: Tablet not found: 30ff3c8fde1a4130a96d8d0ac2b17c27
W20260812 14:20:39.065800 25840 leader_election.cc:341] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81e60aaa3af74dbbb65d148f95028ab4 (127.8.85.3:41981): Not found: Tablet not found: 30ff3c8fde1a4130a96d8d0ac2b17c27
I20260812 14:20:39.065865 25840 leader_election.cc:302] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f27585b67c7146138f8ab173df86daf5; no voters: 81e60aaa3af74dbbb65d148f95028ab4, 8f677095999642a1a2d10ba4066bde70
I20260812 14:20:39.065958 26333 raft_consensus.cc:2726] T 30ff3c8fde1a4130a96d8d0ac2b17c27 P f27585b67c7146138f8ab173df86daf5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:20:39.779762 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:39.779502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.1:54933)
0812 14:20:39.779667 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:39.779714 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260812 14:20:39.811726 26335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:39.811503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.2:34377)
0812 14:20:39.811648 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:39.811681 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260812 14:20:39.943245 26336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:39.943032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33235 (local address 127.8.85.3:46923)
0812 14:20:39.943157 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:39.943200 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:33235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260812 14:20:39.944020 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 25784
I20260812 14:20:39.952536 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 25972
I20260812 14:20:39.961611 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 26124
2026-08-12T14:20:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18415 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:20:40 dist-test-slave-5k54 krb5kdc[26340](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:20:40 dist-test-slave-5k54 krb5kdc[26340](info): set up 2 sockets
Aug 12 14:20:40 dist-test-slave-5k54 krb5kdc[26340](info): commencing operation
krb5kdc: starting...
W20260812 14:20:41.159261 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:20:41 dist-test-slave-5k54 krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:20:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:20:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:20:41.284782 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:44863
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:44863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:41.366973 26354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:41.367177 26354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:41.367198 26354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:41.368556 26354 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 14:20:41.368618 26354 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 14:20:41.368631 26354 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 14:20:41.368642 26354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:20:41.368695 26354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:41.368712 26354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:41.368734 26354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:20:41.368758 26354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:20:41.370203 26354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:44863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:44863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:41.370450 26354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:41.370700 26354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:41.375725 26364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:41.375699 26362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:41.375705 26361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:41.376036 26354 server_base.cc:1034] running on GCE node
I20260812 14:20:41.376271 26354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:41.376514 26354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:41.377657 26354 hybrid_clock.cc:648] HybridClock initialized: now 1786544441377645 us; error 28 us; skew 500 ppm
Aug 12 14:20:41 dist-test-slave-5k54 krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544441, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:41.380190 26354 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:20:41.380658 26354 webserver.cc:460] Webserver started at http://127.8.85.62:41543/ using document root <none> and password file <none>
I20260812 14:20:41.380914 26354 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:41.380985 26354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:41.381135 26354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:20:41.382242 26354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "ed7556c2c93a4517ad510de1ef06e3d0"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "a990f63ccf576d77d9aa6300525d6dc6"
server_key_iv: "c6777239f8b9cef48c52bcbcd5ba55e5"
server_key_version: "encryptionkey@0"
I20260812 14:20:41.382612 26354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "ed7556c2c93a4517ad510de1ef06e3d0"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "a990f63ccf576d77d9aa6300525d6dc6"
server_key_iv: "c6777239f8b9cef48c52bcbcd5ba55e5"
server_key_version: "encryptionkey@0"
I20260812 14:20:41.384068 26354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:20:41.384835 26371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:41.385006 26354 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:20:41.385115 26354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "ed7556c2c93a4517ad510de1ef06e3d0"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "a990f63ccf576d77d9aa6300525d6dc6"
server_key_iv: "c6777239f8b9cef48c52bcbcd5ba55e5"
server_key_version: "encryptionkey@0"
I20260812 14:20:41.385192 26354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:41.407675 26354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:41.409572 26354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:41.409742 26354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:41.413992 26354 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:44863
I20260812 14:20:41.414034 26436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:44863 every 8 connection(s)
I20260812 14:20:41.414405 26354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:20:41.415297 26437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:41.417604 26437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Bootstrap starting.
I20260812 14:20:41.418157 26437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:41.418409 26437 log.cc:824] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:41.419131 26437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: No bootstrap required, opened a new log
I20260812 14:20:41.420146 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 26354
I20260812 14:20:41.420264 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:20:41.420452 8532 external_mini_cluster.cc:1428] Setting key 83badc16e57d475df380492a787747ec
I20260812 14:20:41.421345 26437 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } }
I20260812 14:20:41.421526 26437 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:41.421567 26437 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed7556c2c93a4517ad510de1ef06e3d0, State: Initialized, Role: FOLLOWER
I20260812 14:20:41.421746 26437 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } }
I20260812 14:20:41.421846 26437 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:20:41.421880 26437 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:20:41.421931 26437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:41.422897 26437 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } }
I20260812 14:20:41.422994 26437 leader_election.cc:302] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed7556c2c93a4517ad510de1ef06e3d0; no voters:
I20260812 14:20:41.423175 26437 leader_election.cc:288] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:20:41.423213 26442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:20:41.423389 26442 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 1 LEADER]: Becoming Leader. State: Replica: ed7556c2c93a4517ad510de1ef06e3d0, State: Running, Role: LEADER
I20260812 14:20:41.423480 26437 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:20:41.423544 26442 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } }
I20260812 14:20:41.423914 26442 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed7556c2c93a4517ad510de1ef06e3d0. Latest consensus state: current_term: 1 leader_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } } }
I20260812 14:20:41.424064 26442 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [sys.catalog]: This master's current role is: LEADER
I20260812 14:20:41.424044 26444 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } } }
I20260812 14:20:41.424198 26444 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [sys.catalog]: This master's current role is: LEADER
I20260812 14:20:41.424916 26453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:20:41.425495 26453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:20:41.427057 26453 catalog_manager.cc:1347] Generated new cluster ID: 99918eb5073c464ba87dac70abaaef65
I20260812 14:20:41.427109 26453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:20:41 dist-test-slave-5k54 krb5kdc[26340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:41.432652 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:41.421020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36238 (local address 127.8.85.62:44863)
0812 14:20:41.421220 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:41.421227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:41.421254 (+ 27us) server_negotiation.cc:407] Connection header received
0812 14:20:41.421359 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:41.421364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:41.421407 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:41.421537 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:41.422360 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.423606 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.424712 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.425531 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.425653 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:41.428539 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:41.428543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:41.428543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:41.428555 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:41.430668 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:41.431067 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:41.431071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:41.431071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:41.431202 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:41.431408 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:41.431409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:41.431409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:41.431814 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:41.431999 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:41.432077 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 14:20:41.432379 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260812 14:20:41.437477 26453 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:20:41.438047 26453 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:20:41.445282 26453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Generated new TSK 0
I20260812 14:20:41.445490 26453 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:20:41.482153 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:44863
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:41.584161 26467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:41.584396 26467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:41.584434 26467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:41.585580 26467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:41.585647 26467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:41.585695 26467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:20:41.587168 26467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:44863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:41.587464 26467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:41.587711 26467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:41.592888 26473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:41.592902 26476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:41.592891 26474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:41.593200 26467 server_base.cc:1034] running on GCE node
I20260812 14:20:41.593426 26467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:41.593683 26467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:41.594826 26467 hybrid_clock.cc:648] HybridClock initialized: now 1786544441594804 us; error 36 us; skew 500 ppm
Aug 12 14:20:41 dist-test-slave-5k54 krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544441, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:41.597448 26467 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:20:41.597838 26467 webserver.cc:460] Webserver started at http://127.8.85.1:36299/ using document root <none> and password file <none>
I20260812 14:20:41.598079 26467 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:41.598151 26467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:41.598312 26467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:20:41.599359 26467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "60a1fdec533149d686263a94117a1026"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "28f6d0485e0883e3b284ac757bb901c9"
server_key_iv: "39a63e93bedf06d41c191889f43202ce"
server_key_version: "encryptionkey@0"
I20260812 14:20:41.599810 26467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "60a1fdec533149d686263a94117a1026"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "28f6d0485e0883e3b284ac757bb901c9"
server_key_iv: "39a63e93bedf06d41c191889f43202ce"
server_key_version: "encryptionkey@0"
I20260812 14:20:41.601164 26467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:20:41.601912 26485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:41.602089 26467 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:20:41.602192 26467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "60a1fdec533149d686263a94117a1026"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "28f6d0485e0883e3b284ac757bb901c9"
server_key_iv: "39a63e93bedf06d41c191889f43202ce"
server_key_version: "encryptionkey@0"
I20260812 14:20:41.602275 26467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:41.612459 26467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:41.614742 26467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:41.614912 26467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:41.615144 26467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:41.615520 26467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:41.615576 26467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:41.615618 26467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:41.615644 26467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:41.621361 26467 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:36371
I20260812 14:20:41.621465 26614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:36371 every 8 connection(s)
I20260812 14:20:41.621763 26467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:20:41 dist-test-slave-5k54 krb5kdc[26340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544441, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:41.628575 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 26467
I20260812 14:20:41.628686 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:20:41.628916 8532 external_mini_cluster.cc:1428] Setting key 02dcfa627422a9c998ae865f51932be3
I20260812 14:20:41.632444 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:41.622565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:37523 (local address 127.8.85.62:44863)
0812 14:20:41.622631 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:41.622633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:41.622845 (+ 212us) server_negotiation.cc:407] Connection header received
0812 14:20:41.623410 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:41.623411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:41.623505 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:41.623576 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:41.624691 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.625221 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.626205 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.626402 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.626465 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:41.628913 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:41.628915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:41.628915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:41.628924 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:41.630890 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:41.631298 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:41.631300 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:41.631300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:41.631366 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:41.631668 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:41.631669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:41.631670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:41.631879 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:41.631961 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:41.632307 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 14:20:41.632382 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:20:41.632731 26617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:41.622529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:37523)
0812 14:20:41.622714 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:41.622757 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:41.623314 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:41.623661 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:41.623665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:41.624299 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:41.624598 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:41.624605 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.625304 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.625305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:41.626128 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:41.626130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.626494 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.626495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:41.626601 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:41.627070 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:41.627078 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:41.628805 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:41.630993 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:41.631000 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:41.631002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:41.631220 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:41.631489 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:41.631489 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:41.631492 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:41.631600 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:41.631968 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:41.631977 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:41.632157 (+ 180us) client_negotiation.cc:769] Sending connection context
0812 14:20:41.632280 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 14:20:41.632470 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":111,"threads_started":1}
I20260812 14:20:41.633301 26615 heartbeater.cc:347] Connected to a master server at 127.8.85.62:44863
I20260812 14:20:41.633419 26615 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:41.633597 26615 heartbeater.cc:511] Master 127.8.85.62:44863 requested a full tablet report, sending...
I20260812 14:20:41.634169 26393 ts_manager.cc:194] Registered new tserver with Master: 60a1fdec533149d686263a94117a1026 (127.8.85.1:36371)
I20260812 14:20:41.635185 26393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:37523
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:20:41.656556 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:44863
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:41.739840 26620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:41.740082 26620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:41.740119 26620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:41.741254 26620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:41.741322 26620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:41.741367 26620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:20:41.742815 26620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:44863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:41.743085 26620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:41.743319 26620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:41.748306 26626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:41.748335 26627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:41.748361 26629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:41.748672 26620 server_base.cc:1034] running on GCE node
I20260812 14:20:41.748906 26620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:41.749138 26620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:41.750304 26620 hybrid_clock.cc:648] HybridClock initialized: now 1786544441750294 us; error 40 us; skew 500 ppm
Aug 12 14:20:41 dist-test-slave-5k54 krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544441, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:41.753000 26620 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:20:41.753371 26620 webserver.cc:460] Webserver started at http://127.8.85.2:37311/ using document root <none> and password file <none>
I20260812 14:20:41.753552 26620 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:41.753592 26620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:41.753679 26620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:20:41.754639 26620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "2f2e3ff97abc447b88f6ad61de675ab2"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "c36f77d56668fe1032534fa07c897f9c"
server_key_iv: "791242753dc8936d7558007f983f046b"
server_key_version: "encryptionkey@0"
I20260812 14:20:41.754971 26620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "2f2e3ff97abc447b88f6ad61de675ab2"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "c36f77d56668fe1032534fa07c897f9c"
server_key_iv: "791242753dc8936d7558007f983f046b"
server_key_version: "encryptionkey@0"
I20260812 14:20:41.756341 26620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:20:41.757057 26637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:41.757269 26620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:20:41.757369 26620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "2f2e3ff97abc447b88f6ad61de675ab2"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "c36f77d56668fe1032534fa07c897f9c"
server_key_iv: "791242753dc8936d7558007f983f046b"
server_key_version: "encryptionkey@0"
I20260812 14:20:41.757429 26620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:41.775099 26620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:41.777093 26620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:41.777221 26620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:41.777424 26620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:41.777779 26620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:41.777822 26620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:41.777870 26620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:41.777900 26620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:41.784409 26620 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:44133
I20260812 14:20:41.784497 26770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:44133 every 8 connection(s)
I20260812 14:20:41.784963 26620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:20:41 dist-test-slave-5k54 krb5kdc[26340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544441, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:41.791687 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 26620
I20260812 14:20:41.791779 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:20:41.791986 8532 external_mini_cluster.cc:1428] Setting key e9455dff4c42d43a1879658a56a355b6
I20260812 14:20:41.794973 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:41.785775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:60407 (local address 127.8.85.62:44863)
0812 14:20:41.785845 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:41.785847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:41.786073 (+ 226us) server_negotiation.cc:407] Connection header received
0812 14:20:41.786553 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:41.786554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:41.786565 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:41.786629 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:41.787626 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.788281 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.789378 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.789594 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.789658 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:41.791904 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:41.791905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:41.791906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:41.791916 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:41.793426 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:41.793937 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:41.793938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:41.793938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:41.793989 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:41.794265 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:41.794266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:41.794266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:41.794378 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:41.794427 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:41.794782 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 14:20:41.794872 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:20:41.795150 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:41.785728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:60407)
0812 14:20:41.785914 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:41.785959 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:41.786462 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:41.786695 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:41.786698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:41.787283 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:41.787513 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:41.787518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.788387 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.788387 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:41.789298 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:41.789301 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.789672 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.789673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:41.789768 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:41.790312 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:41.790321 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:41.791752 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:41.793510 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:41.793518 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:41.793521 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:41.793855 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:41.794051 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:41.794052 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:41.794052 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:41.794198 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:41.794443 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:41.794448 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:41.794635 (+ 187us) client_negotiation.cc:769] Sending connection context
0812 14:20:41.794751 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 14:20:41.794910 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260812 14:20:41.795688 26771 heartbeater.cc:347] Connected to a master server at 127.8.85.62:44863
I20260812 14:20:41.795811 26771 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:41.795991 26771 heartbeater.cc:511] Master 127.8.85.62:44863 requested a full tablet report, sending...
I20260812 14:20:41.796492 26393 ts_manager.cc:194] Registered new tserver with Master: 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133)
I20260812 14:20:41.797020 26393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:60407
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:20:41.822181 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:44863
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:20:41.906502 26776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:20:41.906697 26776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:20:41.906723 26776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:20:41.907967 26776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:20:41.908013 26776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:20:41.908041 26776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:20:41.909432 26776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:44863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:20:41.909618 26776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:20:41.909794 26776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:20:41.914839 26787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:20:41.914870 26784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:41.914944 26776 server_base.cc:1034] running on GCE node
W20260812 14:20:41.914868 26781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:20:41.915323 26776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:20:41.915596 26776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:20:41.916746 26776 hybrid_clock.cc:648] HybridClock initialized: now 1786544441916737 us; error 31 us; skew 500 ppm
Aug 12 14:20:41 dist-test-slave-5k54 krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544441, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:20:41.919267 26776 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:20:41.919740 26776 webserver.cc:460] Webserver started at http://127.8.85.3:44017/ using document root <none> and password file <none>
I20260812 14:20:41.919939 26776 fs_manager.cc:359] Metadata directory not provided
I20260812 14:20:41.919981 26776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:20:41.920076 26776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:20:41.921072 26776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "592082633e164ec1953aca0b5ae9e4f7"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "69f63708da72b64145bdedc26850002e"
server_key_iv: "f4d126f611264a7ba1a03ffc770de062"
server_key_version: "encryptionkey@0"
I20260812 14:20:41.921375 26776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "592082633e164ec1953aca0b5ae9e4f7"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "69f63708da72b64145bdedc26850002e"
server_key_iv: "f4d126f611264a7ba1a03ffc770de062"
server_key_version: "encryptionkey@0"
I20260812 14:20:41.922663 26776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:20:41.923496 26795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:41.923686 26776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:20:41.923786 26776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "592082633e164ec1953aca0b5ae9e4f7"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "69f63708da72b64145bdedc26850002e"
server_key_iv: "f4d126f611264a7ba1a03ffc770de062"
server_key_version: "encryptionkey@0"
I20260812 14:20:41.923897 26776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:20:41.969090 26776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:20:41.971019 26776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:20:41.971194 26776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:20:41.971499 26776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:20:41.971843 26776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:20:41.971905 26776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:41.971947 26776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:20:41.971976 26776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:20:41.977864 26776 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:37689
I20260812 14:20:41.977957 26929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:37689 every 8 connection(s)
I20260812 14:20:41.978348 26776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:20:41.978799 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 26776
I20260812 14:20:41.978931 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:20:41.979136 8532 external_mini_cluster.cc:1428] Setting key 43dc1d22f0589c6b6f97c7e8427a2a04
Aug 12 14:20:41 dist-test-slave-5k54 krb5kdc[26340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544441, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:20:41.987635 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:41.979214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:39821 (local address 127.8.85.62:44863)
0812 14:20:41.979261 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:41.979263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:41.979502 (+ 239us) server_negotiation.cc:407] Connection header received
0812 14:20:41.980162 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:41.980162 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:41.980172 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:41.980209 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:41.981133 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.981650 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.982602 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.982789 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.982846 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:41.985022 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:41.985024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:41.985024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:41.985032 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:41.986210 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:41.986613 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:41.986615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:41.986615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:41.986672 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:41.986972 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:41.986973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:41.986973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:41.987085 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:41.987161 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:41.987458 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 14:20:41.987539 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:20:41.987859 26932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:41.979187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:39821)
0812 14:20:41.979367 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:41.979413 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:41.980068 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:41.980268 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:41.980271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:41.980857 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:20:41.981053 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:41.981058 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.981721 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.981721 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:41.982519 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:41.982521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.982866 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.982867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:41.982964 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:41.983402 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:20:41.983411 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:20:41.984925 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:20:41.986292 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:41.986298 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:41.986300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:41.986520 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:41.986736 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:20:41.986737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:20:41.986737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:20:41.986891 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:20:41.987152 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:20:41.987158 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:20:41.987299 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 14:20:41.987411 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 14:20:41.987611 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
I20260812 14:20:41.988349 26930 heartbeater.cc:347] Connected to a master server at 127.8.85.62:44863
I20260812 14:20:41.988468 26930 heartbeater.cc:464] Registering TS with master...
I20260812 14:20:41.988612 26930 heartbeater.cc:511] Master 127.8.85.62:44863 requested a full tablet report, sending...
I20260812 14:20:41.988921 26393 ts_manager.cc:194] Registered new tserver with Master: 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689)
I20260812 14:20:41.989244 26393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:39821
I20260812 14:20:41.990732 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:20:41.997598 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:41.991974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36244 (local address 127.8.85.62:44863)
0812 14:20:41.992029 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:41.992031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:41.992132 (+ 101us) server_negotiation.cc:407] Connection header received
0812 14:20:41.992191 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:41.992192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:41.992199 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:41.992245 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:20:41.992744 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.993175 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.993990 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:41.994152 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:41.994218 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:41.995410 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:20:41.995411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:20:41.995412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:20:41.995417 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:20:41.996643 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:41.997011 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:41.997012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:41.997012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:41.997061 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:20:41.997216 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:20:41.997216 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:20:41.997216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:20:41.997339 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:20:41.997412 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:41.997485 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 14:20:41.997540 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:20:41.999599 26393 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:20:42.001180 26393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:20:42.008731 26947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:42.003740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:37689 (local address 127.0.0.1:32924)
0812 14:20:42.004749 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:42.004756 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:42.004834 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:42.005080 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:42.005081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:42.005083 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:42.005139 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:42.005141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.006444 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.006444 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:42.007753 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:42.007756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.008429 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.008430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:42.008505 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:42.008520 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 14:20:42.008587 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 14:20:42.008642 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"mutex_wait_us":51,"thread_start_us":133,"threads_started":1}
I20260812 14:20:42.008872 26950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:42.003727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:44133 (local address 127.0.0.1:33790)
0812 14:20:42.004387 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:42.004427 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:42.004523 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:42.004786 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:42.004790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:42.004792 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:42.004861 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:42.004863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.006321 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.006322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:42.007571 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:42.007575 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.008538 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.008539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:42.008614 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:42.008619 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:20:42.008669 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 14:20:42.008716 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":42,"thread_start_us":148,"threads_started":1}
I20260812 14:20:42.009002 26952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:42.004027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32924 (local address 127.8.85.3:37689)
0812 14:20:42.004199 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:42.004203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:42.004835 (+ 632us) server_negotiation.cc:407] Connection header received
0812 14:20:42.004898 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:42.004901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:42.004919 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:42.004996 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:42.005222 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.006333 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.007864 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.008339 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.008428 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:42.008490 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:42.008858 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 14:20:42.008907 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260812 14:20:42.009156 26948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:42.003731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:36371 (local address 127.0.0.1:52498)
0812 14:20:42.004636 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:42.004644 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:42.004686 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:42.005043 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:42.005043 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:42.005045 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:42.005076 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:42.005077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.006443 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.006444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:42.007753 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:42.007756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.008582 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.008583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:42.008636 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:42.008639 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:20:42.008686 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 14:20:42.008720 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"mutex_wait_us":43,"thread_start_us":62,"threads_started":1}
I20260812 14:20:42.009194 26953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:42.004172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33790 (local address 127.8.85.2:44133)
0812 14:20:42.004349 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:42.004354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:42.004503 (+ 149us) server_negotiation.cc:407] Connection header received
0812 14:20:42.004642 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:42.004646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:42.004664 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:42.004750 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:42.005531 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.006202 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.007666 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.008407 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.008965 (+ 558us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:42.009035 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:42.009061 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 14:20:42.009098 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
I20260812 14:20:42.009816 26851 tablet_service.cc:1467] Processing CreateTablet for tablet 71c5a688ffd3467fb1caec09c01d726f (DEFAULT_TABLE table=test-table [id=d05b2c8afb09495289ac7f4e390a9548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:20:42.009814 26690 tablet_service.cc:1467] Processing CreateTablet for tablet 71c5a688ffd3467fb1caec09c01d726f (DEFAULT_TABLE table=test-table [id=d05b2c8afb09495289ac7f4e390a9548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:20:42.010200 26690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71c5a688ffd3467fb1caec09c01d726f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:42.010200 26851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71c5a688ffd3467fb1caec09c01d726f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:42.011570 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:42.003938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52498 (local address 127.8.85.1:36371)
0812 14:20:42.004161 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:42.004167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:42.004894 (+ 727us) server_negotiation.cc:407] Connection header received
0812 14:20:42.004941 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:42.004945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:42.004960 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:42.005033 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:42.005124 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.006277 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.007840 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.008515 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.008581 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:42.008642 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:42.008699 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 14:20:42.008737 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":105,"threads_started":1}
I20260812 14:20:42.012341 26540 tablet_service.cc:1467] Processing CreateTablet for tablet 71c5a688ffd3467fb1caec09c01d726f (DEFAULT_TABLE table=test-table [id=d05b2c8afb09495289ac7f4e390a9548]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:20:42.012686 26540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71c5a688ffd3467fb1caec09c01d726f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:20:42.013640 26956 tablet_bootstrap.cc:492] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2: Bootstrap starting.
I20260812 14:20:42.014423 26956 tablet_bootstrap.cc:654] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:42.015056 26956 log.cc:824] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:42.015487 26954 tablet_bootstrap.cc:492] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7: Bootstrap starting.
I20260812 14:20:42.015895 26956 tablet_bootstrap.cc:492] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2: No bootstrap required, opened a new log
I20260812 14:20:42.015997 26956 ts_tablet_manager.cc:1397] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:20:42.015914 26958 tablet_bootstrap.cc:492] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Bootstrap starting.
I20260812 14:20:42.016239 26954 tablet_bootstrap.cc:654] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:42.016656 26958 tablet_bootstrap.cc:654] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Neither blocks nor log segments found. Creating new log.
I20260812 14:20:42.016695 26954 log.cc:824] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:42.017231 26958 log.cc:824] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Log is configured to *not* fsync() on all Append() calls
I20260812 14:20:42.017542 26954 tablet_bootstrap.cc:492] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7: No bootstrap required, opened a new log
I20260812 14:20:42.017622 26954 ts_tablet_manager.cc:1397] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:20:42.018008 26958 tablet_bootstrap.cc:492] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: No bootstrap required, opened a new log
I20260812 14:20:42.018083 26958 ts_tablet_manager.cc:1397] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:20:42.018261 26956 raft_consensus.cc:348] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } } peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } }
I20260812 14:20:42.018440 26956 raft_consensus.cc:374] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:42.018476 26956 raft_consensus.cc:729] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2e3ff97abc447b88f6ad61de675ab2, State: Initialized, Role: FOLLOWER
I20260812 14:20:42.018615 26956 consensus_queue.cc:260] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } } peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } }
I20260812 14:20:42.018903 26956 ts_tablet_manager.cc:1428] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:20:42.019516 26771 heartbeater.cc:503] Master 127.8.85.62:44863 was elected leader, sending a full tablet report...
I20260812 14:20:42.019533 26954 raft_consensus.cc:348] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } } peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } }
I20260812 14:20:42.019719 26954 raft_consensus.cc:374] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:42.019760 26954 raft_consensus.cc:729] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 592082633e164ec1953aca0b5ae9e4f7, State: Initialized, Role: FOLLOWER
I20260812 14:20:42.019917 26954 consensus_queue.cc:260] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } } peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } }
I20260812 14:20:42.020205 26954 ts_tablet_manager.cc:1428] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:20:42.020314 26930 heartbeater.cc:503] Master 127.8.85.62:44863 was elected leader, sending a full tablet report...
I20260812 14:20:42.020253 26958 raft_consensus.cc:348] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } } peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } }
I20260812 14:20:42.020427 26958 raft_consensus.cc:374] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:20:42.020483 26958 raft_consensus.cc:729] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60a1fdec533149d686263a94117a1026, State: Initialized, Role: FOLLOWER
I20260812 14:20:42.020602 26958 consensus_queue.cc:260] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } } peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } }
I20260812 14:20:42.020915 26958 ts_tablet_manager.cc:1428] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:20:42.020915 26615 heartbeater.cc:503] Master 127.8.85.62:44863 was elected leader, sending a full tablet report...
W20260812 14:20:42.035614 26772 tablet.cc:2369] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:20:42.122494 26616 tablet.cc:2369] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:20:42.219646 26963 raft_consensus.cc:482] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:20:42.219789 26963 raft_consensus.cc:504] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } } peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } }
I20260812 14:20:42.220211 26963 leader_election.cc:288] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133), 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689)
I20260812 14:20:42.223330 26953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:42.220586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:38883 (local address 127.8.85.2:44133)
0812 14:20:42.220658 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:42.220660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:42.220756 (+ 96us) server_negotiation.cc:407] Connection header received
0812 14:20:42.220816 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:42.220817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:42.220871 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:42.220937 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:42.221157 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.221596 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.222606 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.222976 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.223061 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:42.223074 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:42.223214 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 14:20:42.223262 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 14:20:42.223341 26964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:42.220432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:44133 (local address 127.8.85.1:38883)
0812 14:20:42.220645 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:42.220665 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:42.220754 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:42.220975 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:42.220976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:42.220978 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:42.221074 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:42.221076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.221702 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.221703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:42.222507 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:42.222508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.223057 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.223059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:42.223132 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:42.223144 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 14:20:42.223187 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 14:20:42.223218 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":17,"thread_start_us":68,"threads_started":1}
I20260812 14:20:42.223414 26952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:42.220545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:45065 (local address 127.8.85.3:37689)
0812 14:20:42.220608 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:42.220610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:42.220716 (+ 106us) server_negotiation.cc:407] Connection header received
0812 14:20:42.220804 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:42.220805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:42.220820 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:42.220873 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:20:42.221075 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.221585 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.222604 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.222978 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.223066 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:42.223075 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:42.223297 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 14:20:42.223340 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:20:42.223646 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:42.220432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:37689 (local address 127.8.85.1:45065)
0812 14:20:42.220606 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:42.220621 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 14:20:42.220715 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:20:42.220897 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:20:42.220898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:20:42.220901 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:20:42.220991 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:42.220994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.221670 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.221671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:42.222506 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:20:42.222508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.223119 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.223120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:20:42.223210 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:42.223214 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:20:42.223270 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 14:20:42.223306 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":34,"thread_start_us":74,"threads_started":1}
I20260812 14:20:42.223802 26715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c5a688ffd3467fb1caec09c01d726f" candidate_uuid: "60a1fdec533149d686263a94117a1026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" is_pre_election: true
I20260812 14:20:42.223990 26715 raft_consensus.cc:2454] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60a1fdec533149d686263a94117a1026 in term 0.
I20260812 14:20:42.223994 26874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c5a688ffd3467fb1caec09c01d726f" candidate_uuid: "60a1fdec533149d686263a94117a1026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "592082633e164ec1953aca0b5ae9e4f7" is_pre_election: true
I20260812 14:20:42.224126 26874 raft_consensus.cc:2454] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60a1fdec533149d686263a94117a1026 in term 0.
I20260812 14:20:42.224236 26487 leader_election.cc:302] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f2e3ff97abc447b88f6ad61de675ab2, 60a1fdec533149d686263a94117a1026; no voters:
I20260812 14:20:42.224391 26963 raft_consensus.cc:2781] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:20:42.224467 26963 raft_consensus.cc:482] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:20:42.224503 26963 raft_consensus.cc:3037] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:42.225419 26963 raft_consensus.cc:504] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } } peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } }
I20260812 14:20:42.225576 26963 leader_election.cc:288] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 1 election: Requested vote from peers 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133), 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689)
I20260812 14:20:42.225720 26715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c5a688ffd3467fb1caec09c01d726f" candidate_uuid: "60a1fdec533149d686263a94117a1026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f2e3ff97abc447b88f6ad61de675ab2"
I20260812 14:20:42.225762 26874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71c5a688ffd3467fb1caec09c01d726f" candidate_uuid: "60a1fdec533149d686263a94117a1026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "592082633e164ec1953aca0b5ae9e4f7"
I20260812 14:20:42.225857 26874 raft_consensus.cc:3037] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:42.225819 26715 raft_consensus.cc:3037] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:20:42.226588 26874 raft_consensus.cc:2454] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60a1fdec533149d686263a94117a1026 in term 1.
I20260812 14:20:42.226591 26715 raft_consensus.cc:2454] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60a1fdec533149d686263a94117a1026 in term 1.
I20260812 14:20:42.226776 26488 leader_election.cc:302] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 592082633e164ec1953aca0b5ae9e4f7, 60a1fdec533149d686263a94117a1026; no voters:
I20260812 14:20:42.226890 26963 raft_consensus.cc:2781] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:20:42.227072 26963 raft_consensus.cc:686] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 1 LEADER]: Becoming Leader. State: Replica: 60a1fdec533149d686263a94117a1026, State: Running, Role: LEADER
I20260812 14:20:42.227177 26963 consensus_queue.cc:237] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } } peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } }
I20260812 14:20:42.228114 26391 catalog_manager.cc:5495] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60a1fdec533149d686263a94117a1026 (127.8.85.1). New cstate: current_term: 1 leader_uuid: "60a1fdec533149d686263a94117a1026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } health_report { overall_health: UNKNOWN } } }
W20260812 14:20:42.228993 26931 tablet.cc:2369] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:20:42.235410 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:42.232959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52514 (local address 127.8.85.1:36371)
0812 14:20:42.233022 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:20:42.233024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:20:42.233033 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:20:42.233064 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:20:42.233065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:20:42.233085 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:20:42.233173 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:20:42.233306 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.233795 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.234595 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:20:42.234812 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:20:42.234905 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:20:42.235085 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:20:42.235187 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:20:42.235297 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 14:20:42.235315 (+ 18us) server_negotiation.cc:299] Negotiation successful
0812 14:20:42.235334 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:20:42.237123 26874 raft_consensus.cc:1261] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7 [term 1 FOLLOWER]: Refusing update from remote peer 60a1fdec533149d686263a94117a1026: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:20:42.237155 26715 raft_consensus.cc:1261] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2 [term 1 FOLLOWER]: Refusing update from remote peer 60a1fdec533149d686263a94117a1026: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:20:42.237509 26967 consensus_queue.cc:1035] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:20:42.237648 26963 consensus_queue.cc:1035] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:20:42.240233 26973 mvcc.cc:205] Tried to move back new op lower bound from 7317686035400835072 to 7317686035363123200. Current Snapshot: MvccSnapshot[applied={T|T < 7317686035400835072}]
I20260812 14:20:42.240458 26971 mvcc.cc:205] Tried to move back new op lower bound from 7317686035400835072 to 7317686035363123200. Current Snapshot: MvccSnapshot[applied={T|T < 7317686035400835072}]
I20260812 14:20:42.242653 26393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:20:42.242755 26393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:20:42.243546 26393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Sending DeleteTablet for 3 replicas of tablet 71c5a688ffd3467fb1caec09c01d726f
I20260812 14:20:42.243896 26540 tablet_service.cc:1514] Processing DeleteTablet for tablet 71c5a688ffd3467fb1caec09c01d726f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:20:42 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:52498
I20260812 14:20:42.244158 26983 tablet_replica.cc:330] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: stopping tablet replica
I20260812 14:20:42.244226 26851 tablet_service.cc:1514] Processing DeleteTablet for tablet 71c5a688ffd3467fb1caec09c01d726f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:20:42 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:32924
I20260812 14:20:42.244313 26983 raft_consensus.cc:2229] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:20:42.244395 26690 tablet_service.cc:1514] Processing DeleteTablet for tablet 71c5a688ffd3467fb1caec09c01d726f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:20:42 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:33790
I20260812 14:20:42.244463 26983 raft_consensus.cc:2258] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:20:42.244632 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 26467
I20260812 14:20:42.244905 26983 ts_tablet_manager.cc:1905] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:20:42.248160 26985 tablet_replica.cc:330] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2: stopping tablet replica
I20260812 14:20:42.248289 26985 raft_consensus.cc:2229] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:20:42.248404 26985 raft_consensus.cc:2258] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:20:42.248796 26984 tablet_replica.cc:330] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7: stopping tablet replica
I20260812 14:20:42.248916 26984 raft_consensus.cc:2229] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:20:42.248920 26985 ts_tablet_manager.cc:1905] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:20:42.249024 26984 raft_consensus.cc:2258] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:20:42.249537 26984 ts_tablet_manager.cc:1905] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:20:42.250916 26985 ts_tablet_manager.cc:1918] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:20:42.250999 26985 log.cc:1196] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/71c5a688ffd3467fb1caec09c01d726f
I20260812 14:20:42.251312 26985 ts_tablet_manager.cc:1939] T 71c5a688ffd3467fb1caec09c01d726f P 2f2e3ff97abc447b88f6ad61de675ab2: Deleting consensus metadata
I20260812 14:20:42.251507 26984 ts_tablet_manager.cc:1918] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:20:42.251658 26984 log.cc:1196] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/71c5a688ffd3467fb1caec09c01d726f
I20260812 14:20:42.251946 26984 ts_tablet_manager.cc:1939] T 71c5a688ffd3467fb1caec09c01d726f P 592082633e164ec1953aca0b5ae9e4f7: Deleting consensus metadata
I20260812 14:20:42.252425 26373 catalog_manager.cc:4841] TS 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133): tablet 71c5a688ffd3467fb1caec09c01d726f (table test-table [id=d05b2c8afb09495289ac7f4e390a9548]) successfully deleted
I20260812 14:20:42.252501 26375 catalog_manager.cc:4841] TS 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689): tablet 71c5a688ffd3467fb1caec09c01d726f (table test-table [id=d05b2c8afb09495289ac7f4e390a9548]) successfully deleted
W20260812 14:20:42.254351 26372 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:36371 (error 108)
I20260812 14:20:42.254696 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 26620
I20260812 14:20:42.254886 26948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:20:42.254742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:36371 (local address 127.0.0.1:52524)
0812 14:20:42.254808 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 14:20:42.254847 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:36371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 14:20:42.258567 26372 catalog_manager.cc:4584] TS 60a1fdec533149d686263a94117a1026 (127.8.85.1:36371): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 71c5a688ffd3467fb1caec09c01d726f: Network error: Client connection negotiation failed: client connection to 127.8.85.1:36371: connect: Connection refused (error 111)
I20260812 14:20:42.262261 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 26776
I20260812 14:20:42.269115 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 26354
I20260812 14:21:02.276842 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:44863
--webserver_interface=127.8.85.62
--webserver_port=41543
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:44863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:02.360184 26987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:02.360376 26987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:02.360396 26987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:02.361524 26987 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 14:21:02.361564 26987 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 14:21:02.361578 26987 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 14:21:02.361590 26987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:02.361603 26987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:02.361653 26987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:02.361680 26987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:02.361703 26987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:02.363227 26987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:44863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:44863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=41543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:02.363495 26987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:02.363767 26987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:02.373342 26998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:02.373383 26987 server_base.cc:1034] running on GCE node
W20260812 14:21:02.373385 26993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:02.373348 26996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:02.373796 26987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:02.374135 26987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:02.375284 26987 hybrid_clock.cc:648] HybridClock initialized: now 1786544462375267 us; error 39 us; skew 500 ppm
Aug 12 14:21:02 dist-test-slave-5k54 krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544462, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:02.384235 26987 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:21:02.384748 26987 webserver.cc:460] Webserver started at http://127.8.85.62:41543/ using document root <none> and password file <none>
I20260812 14:21:02.385011 26987 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:02.385095 26987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:02.386485 26987 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:21:02.387167 27006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:02.387336 26987 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:02.387461 26987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "ed7556c2c93a4517ad510de1ef06e3d0"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "a990f63ccf576d77d9aa6300525d6dc6"
server_key_iv: "c6777239f8b9cef48c52bcbcd5ba55e5"
server_key_version: "encryptionkey@0"
I20260812 14:21:02.387890 26987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:02.395992 26987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:02.398231 26987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:02.398383 26987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:02.402473 26987 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:44863
I20260812 14:21:02.402508 27066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:44863 every 8 connection(s)
I20260812 14:21:02.402948 26987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:02.404466 27068 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:21:02.405107 27068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Bootstrap starting.
I20260812 14:21:02.413120 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 26987
I20260812 14:21:02.413452 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:36371
--local_ip_for_outbound_sockets=127.8.85.1
--tserver_master_addrs=127.8.85.62:44863
--webserver_port=36299
--webserver_interface=127.8.85.1
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:21:02.413635 27068 log.cc:824] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:02.415940 27068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:21:02.416225 27068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Bootstrap complete.
I20260812 14:21:02.418644 27068 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } }
I20260812 14:21:02.418867 27068 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed7556c2c93a4517ad510de1ef06e3d0, State: Initialized, Role: FOLLOWER
I20260812 14:21:02.419023 27068 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } }
I20260812 14:21:02.419093 27068 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:02.419153 27068 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:02.419203 27068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:21:02.420295 27068 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } }
I20260812 14:21:02.420384 27068 leader_election.cc:302] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed7556c2c93a4517ad510de1ef06e3d0; no voters:
I20260812 14:21:02.420590 27068 leader_election.cc:288] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 14:21:02.420625 27073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 14:21:02.420742 27073 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 2 LEADER]: Becoming Leader. State: Replica: ed7556c2c93a4517ad510de1ef06e3d0, State: Running, Role: LEADER
I20260812 14:21:02.420866 27068 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:02.420840 27073 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } }
I20260812 14:21:02.421329 27074 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } } }
I20260812 14:21:02.421442 27074 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:02.421648 27077 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed7556c2c93a4517ad510de1ef06e3d0. Latest consensus state: current_term: 2 leader_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } } }
I20260812 14:21:02.421729 27077 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:02.421849 27083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:02.422720 27083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:02.422899 27083 catalog_manager.cc:1259] Loaded cluster ID: 99918eb5073c464ba87dac70abaaef65
I20260812 14:21:02.423079 27083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:21:02.424499 27083 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:02.424705 27083 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Loaded TSK: 0
I20260812 14:21:02.437186 27083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Generated new TSK 1
I20260812 14:21:02.437280 27083 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 14:21:02.504190 27072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:02.504423 27072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:02.504451 27072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:02.505750 27072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:02.505797 27072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:02.505831 27072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:21:02.507489 27072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:36371
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=36299
--tserver_master_addrs=127.8.85.62:44863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:02.507754 27072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:02.508036 27072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:02.514315 27101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:02.514315 27102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:02.514655 27104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:02.514652 27072 server_base.cc:1034] running on GCE node
I20260812 14:21:02.514927 27072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:02.515198 27072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:02.516386 27072 hybrid_clock.cc:648] HybridClock initialized: now 1786544462516372 us; error 29 us; skew 500 ppm
Aug 12 14:21:02 dist-test-slave-5k54 krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544462, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:02.519579 27072 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:21:02.520113 27072 webserver.cc:460] Webserver started at http://127.8.85.1:36299/ using document root <none> and password file <none>
I20260812 14:21:02.520357 27072 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:02.520444 27072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:02.521930 27072 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:21:02.522856 27113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:02.523075 27072 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:21:02.523177 27072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "60a1fdec533149d686263a94117a1026"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "28f6d0485e0883e3b284ac757bb901c9"
server_key_iv: "39a63e93bedf06d41c191889f43202ce"
server_key_version: "encryptionkey@0"
I20260812 14:21:02.523692 27072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:02.541627 27072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:02.544088 27072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:02.544241 27072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:02.544508 27072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:02.545003 27120 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 14:21:02.546519 27072 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 14:21:02.546582 27072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 14:21:02.546617 27072 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 14:21:02.547566 27072 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 14:21:02.547613 27072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260812 14:21:02.547709 27120 tablet_bootstrap.cc:492] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Bootstrap starting.
I20260812 14:21:02.549465 27120 log.cc:824] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:02.552699 27120 tablet_bootstrap.cc:492] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:21:02.553017 27120 tablet_bootstrap.cc:492] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Bootstrap complete.
I20260812 14:21:02.553180 27120 ts_tablet_manager.cc:1397] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 14:21:02.554854 27072 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:36371
I20260812 14:21:02.555213 27072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:21:02.555027 27120 raft_consensus.cc:348] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } } peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } }
I20260812 14:21:02.555321 27120 raft_consensus.cc:729] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60a1fdec533149d686263a94117a1026, State: Initialized, Role: FOLLOWER
I20260812 14:21:02.555613 27120 consensus_queue.cc:260] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } } peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } }
I20260812 14:21:02.555902 27120 ts_tablet_manager.cc:1428] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 14:21:02.556222 27245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:36371 every 8 connection(s)
I20260812 14:21:02.559970 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 27072
I20260812 14:21:02.560258 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:44133
--local_ip_for_outbound_sockets=127.8.85.2
--tserver_master_addrs=127.8.85.62:44863
--webserver_port=37311
--webserver_interface=127.8.85.2
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:21:02 dist-test-slave-5k54 krb5kdc[26340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544462, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:21:02.568329 27249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.556787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:40185)
0812 14:21:02.556951 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:02.556997 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:02.557136 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:02.557547 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:02.557550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:02.557817 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:02.558008 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.558012 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.559564 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.559565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.560533 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.560536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.561223 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.561224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.561351 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.561941 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:21:02.561953 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:02.564307 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:02.566645 (+ 2338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:02.566652 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:02.566654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:02.566901 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:02.567200 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:02.567200 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:02.567201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:02.567316 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:02.567835 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:02.567841 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:02.567955 (+ 114us) client_negotiation.cc:769] Sending connection context
0812 14:21:02.568041 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 14:21:02.568161 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
I20260812 14:21:02.568526 27251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.556859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:40185 (local address 127.8.85.62:44863)
0812 14:21:02.557041 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:02.557049 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:02.557090 (+ 41us) server_negotiation.cc:407] Connection header received
0812 14:21:02.557316 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:02.557321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:02.557372 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:02.557516 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:02.558127 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.559451 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.560633 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.561125 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.561249 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.564413 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:02.564418 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:02.564418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:02.564430 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:02.566558 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:02.566990 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:02.566993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:02.566993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:02.567124 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:02.567393 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:02.567394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:02.567394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:02.567747 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:02.567838 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:02.568135 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 14:21:02.568295 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260812 14:21:02.569168 27246 heartbeater.cc:347] Connected to a master server at 127.8.85.62:44863
I20260812 14:21:02.569289 27246 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:02.569475 27246 heartbeater.cc:511] Master 127.8.85.62:44863 requested a full tablet report, sending...
I20260812 14:21:02.570241 27025 ts_manager.cc:194] Registered new tserver with Master: 60a1fdec533149d686263a94117a1026 (127.8.85.1:36371)
I20260812 14:21:02.571327 27025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:40185
Aug 12 14:21:02 dist-test-slave-5k54 krb5kdc[26340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544462, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for kudu/127.8.85.1@KRBTEST.COM
I20260812 14:21:02.579852 27254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.570835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:36371 (local address 127.0.0.1:45808)
0812 14:21:02.571680 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:02.571713 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:02.571803 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:02.572022 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:02.572028 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:02.572348 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:02.572402 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.572404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.573733 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.573734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.574321 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.574323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.575001 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.575004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.575057 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.575617 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:21:02.575621 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:02.576943 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:02.578775 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:02.578782 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:02.578783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:02.578962 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:02.579208 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:02.579208 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:02.579208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:02.579259 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:02.579567 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:02.579573 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:02.579616 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 14:21:02.579683 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 14:21:02.579772 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"thread_start_us":89,"threads_started":1}
I20260812 14:21:02.579932 27255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.570926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45808 (local address 127.8.85.1:36371)
0812 14:21:02.571072 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:02.571076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:02.571787 (+ 711us) server_negotiation.cc:407] Connection header received
0812 14:21:02.571912 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:02.571915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:02.571931 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:02.572010 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:02.572494 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.573649 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.574404 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.574913 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.574984 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.577031 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:02.577038 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:02.577039 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:02.577045 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:02.578700 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:02.579056 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:02.579060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:02.579060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:02.579131 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:02.579320 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:02.579322 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:02.579322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:02.579502 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:02.579589 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:02.579735 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 14:21:02.579819 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260812 14:21:02.580296 27169 tablet_service.cc:1514] Processing DeleteTablet for tablet 71c5a688ffd3467fb1caec09c01d726f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:20:42 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:45808
I20260812 14:21:02.580893 27256 tablet_replica.cc:330] stopping tablet replica
I20260812 14:21:02.581003 27256 raft_consensus.cc:2229] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:21:02.581074 27256 raft_consensus.cc:2258] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:02.581354 27256 ts_tablet_manager.cc:1905] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:21:02.583714 27256 ts_tablet_manager.cc:1918] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:21:02.583777 27256 log.cc:1196] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal/wals/71c5a688ffd3467fb1caec09c01d726f
I20260812 14:21:02.584017 27256 ts_tablet_manager.cc:1939] T 71c5a688ffd3467fb1caec09c01d726f P 60a1fdec533149d686263a94117a1026: Deleting consensus metadata
I20260812 14:21:02.584513 27007 catalog_manager.cc:4841] TS 60a1fdec533149d686263a94117a1026 (127.8.85.1:36371): tablet 71c5a688ffd3467fb1caec09c01d726f (table test-table [id=d05b2c8afb09495289ac7f4e390a9548]) successfully deleted
W20260812 14:21:02.648591 27253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:02.648847 27253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:02.648886 27253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:02.650019 27253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:02.650089 27253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:02.650133 27253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:21:02.651598 27253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:44133
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=37311
--tserver_master_addrs=127.8.85.62:44863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:02.651878 27253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:02.652112 27253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:02.657816 27263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:02.657874 27253 server_base.cc:1034] running on GCE node
W20260812 14:21:02.657881 27265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:02.657816 27262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:02.658224 27253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:02.658458 27253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:02.659631 27253 hybrid_clock.cc:648] HybridClock initialized: now 1786544462659618 us; error 27 us; skew 500 ppm
Aug 12 14:21:02 dist-test-slave-5k54 krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544462, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:02.661993 27253 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:21:02.662406 27253 webserver.cc:460] Webserver started at http://127.8.85.2:37311/ using document root <none> and password file <none>
I20260812 14:21:02.662670 27253 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:02.662746 27253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:02.664220 27253 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:21:02.664740 27272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:02.664925 27253 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:02.664992 27253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "2f2e3ff97abc447b88f6ad61de675ab2"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "c36f77d56668fe1032534fa07c897f9c"
server_key_iv: "791242753dc8936d7558007f983f046b"
server_key_version: "encryptionkey@0"
I20260812 14:21:02.665287 27253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:02.674492 27253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:02.676772 27253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:02.676911 27253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:02.677135 27253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:02.677431 27253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:02.677489 27253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:02.677531 27253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:02.677559 27253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:02.683804 27253 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:44133
I20260812 14:21:02.683892 27407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:44133 every 8 connection(s)
I20260812 14:21:02.684363 27253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:21:02.686685 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 27253
I20260812 14:21:02.687045 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:37689
--local_ip_for_outbound_sockets=127.8.85.3
--tserver_master_addrs=127.8.85.62:44863
--webserver_port=44017
--webserver_interface=127.8.85.3
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:21:02 dist-test-slave-5k54 krb5kdc[26340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544462, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:21:02.696058 27251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.685184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:60641 (local address 127.8.85.62:44863)
0812 14:21:02.685231 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:02.685233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:02.685492 (+ 259us) server_negotiation.cc:407] Connection header received
0812 14:21:02.686094 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:02.686096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:02.686116 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:02.686180 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:02.687661 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.688520 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.689950 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.690200 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.690294 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.692911 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:02.692913 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:02.692913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:02.692924 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:02.694373 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:02.694904 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:02.694906 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:02.694906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:02.694977 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:02.695268 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:02.695269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:02.695269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:02.695402 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:02.695518 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:02.695843 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 14:21:02.695955 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:21:02.696250 27410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.685182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:60641)
0812 14:21:02.685359 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:02.685397 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:02.685974 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:02.686253 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:02.686257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:02.687185 (+ 928us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:02.687527 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.687535 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.688658 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.688659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.689746 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.689748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.690325 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.690327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.690460 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.691070 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:21:02.691079 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:02.692799 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:02.694481 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:02.694489 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:02.694492 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:02.694817 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:02.695059 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:02.695060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:02.695060 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:02.695184 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:02.695530 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:02.695538 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:02.695687 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 14:21:02.695808 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 14:21:02.695984 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260812 14:21:02.696906 27408 heartbeater.cc:347] Connected to a master server at 127.8.85.62:44863
I20260812 14:21:02.697041 27408 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:02.697244 27408 heartbeater.cc:511] Master 127.8.85.62:44863 requested a full tablet report, sending...
I20260812 14:21:02.697701 27025 ts_manager.cc:194] Registered new tserver with Master: 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133)
I20260812 14:21:02.698127 27025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:60641
W20260812 14:21:02.790884 27411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:02.791088 27411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:02.791112 27411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:02.792385 27411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:02.792428 27411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:02.792476 27411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:21:02.793941 27411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:37689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=44017
--tserver_master_addrs=127.8.85.62:44863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:02.794185 27411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:02.794423 27411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:02.799532 27417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:02.799497 27418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:02.799517 27420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:02.799691 27411 server_base.cc:1034] running on GCE node
I20260812 14:21:02.799894 27411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:02.800149 27411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:02.801308 27411 hybrid_clock.cc:648] HybridClock initialized: now 1786544462801298 us; error 41 us; skew 500 ppm
Aug 12 14:21:02 dist-test-slave-5k54 krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544462, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:02.803902 27411 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:21:02.804334 27411 webserver.cc:460] Webserver started at http://127.8.85.3:44017/ using document root <none> and password file <none>
I20260812 14:21:02.804538 27411 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:02.804593 27411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:02.805908 27411 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:21:02.806453 27428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:02.806609 27411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:02.806703 27411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "592082633e164ec1953aca0b5ae9e4f7"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "69f63708da72b64145bdedc26850002e"
server_key_iv: "f4d126f611264a7ba1a03ffc770de062"
server_key_version: "encryptionkey@0"
I20260812 14:21:02.807029 27411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:02.816545 27411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:02.818768 27411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:02.818892 27411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:02.819065 27411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:02.819379 27411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:02.819415 27411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:02.819495 27411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:02.819527 27411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:02.825528 27411 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:37689
I20260812 14:21:02.825610 27558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:37689 every 8 connection(s)
I20260812 14:21:02.825950 27411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:21:02 dist-test-slave-5k54 krb5kdc[26340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544462, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:21:02.834570 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 27411
I20260812 14:21:02.835352 27251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.826675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:57195 (local address 127.8.85.62:44863)
0812 14:21:02.826745 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:02.826747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:02.826949 (+ 202us) server_negotiation.cc:407] Connection header received
0812 14:21:02.827568 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:02.827569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:02.827582 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:02.827637 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:02.828648 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.829155 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.830196 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.830412 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.830487 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.832521 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:02.832522 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:02.832523 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:02.832530 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:02.833957 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:02.834303 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:02.834304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:02.834304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:02.834382 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:02.834613 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:02.834614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:02.834614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:02.834770 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:02.834884 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:02.835203 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 14:21:02.835281 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 14:21:02.835608 27562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.826660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:57195)
0812 14:21:02.826833 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:02.826868 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:02.827449 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:02.827706 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:02.827715 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:02.828322 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:02.828554 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.828559 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.829265 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.829266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.830098 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.830101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.830511 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.830512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.830640 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.831067 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:21:02.831075 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:02.832432 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:02.834029 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:02.834034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:02.834036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:02.834236 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:02.834456 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:02.834456 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:02.834457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:02.834548 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:02.834849 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:02.834856 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:02.835046 (+ 190us) client_negotiation.cc:769] Sending connection context
0812 14:21:02.835171 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 14:21:02.835326 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
I20260812 14:21:02.836119 27559 heartbeater.cc:347] Connected to a master server at 127.8.85.62:44863
I20260812 14:21:02.836229 27559 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:02.836442 27559 heartbeater.cc:511] Master 127.8.85.62:44863 requested a full tablet report, sending...
I20260812 14:21:02.836813 27025 ts_manager.cc:194] Registered new tserver with Master: 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689)
I20260812 14:21:02.837348 27025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:57195
I20260812 14:21:02.842017 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.835065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35706 (local address 127.8.85.62:44863)
0812 14:21:02.835182 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:02.835184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:02.835829 (+ 645us) server_negotiation.cc:407] Connection header received
0812 14:21:02.835904 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:02.835905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:02.835919 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:02.835987 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:02.836761 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.837543 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.838492 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.838693 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.838747 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.839862 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:02.839864 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:02.839864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:02.839870 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:02.841067 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:02.841379 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:02.841380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:02.841380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:02.841431 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:02.841603 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:02.841604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:02.841604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:02.841699 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:02.841821 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:02.841850 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 14:21:02.841912 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":56,"threads_started":1}
I20260812 14:21:02.842743 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:21:02.845559 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.843091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35710 (local address 127.8.85.62:44863)
0812 14:21:02.843162 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:02.843164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:02.843258 (+ 94us) server_negotiation.cc:407] Connection header received
0812 14:21:02.843336 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:02.843337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:02.843374 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:02.843475 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:21:02.843597 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.844056 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.844826 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.845042 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.845116 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.845278 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:21:02.845330 (+ 52us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 14:21:02.845491 (+ 161us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35710: token expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 14:21:02.845618 27564 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35710: token expired
W20260812 14:21:02.846418 26938 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.85.62:44863)
I20260812 14:21:02.852195 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.846709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35722 (local address 127.8.85.62:44863)
0812 14:21:02.846761 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:02.846763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:02.846770 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:21:02.846792 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:02.846793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:02.846801 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:02.846849 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:02.847301 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.847804 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.848470 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.848676 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.848760 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.849867 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:02.849868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:02.849869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:02.849875 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:02.851102 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:02.851458 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:02.851459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:02.851459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:02.851512 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:02.851718 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:02.851719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:02.851719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:02.851826 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:02.851923 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:02.852028 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 14:21:02.852101 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:21:02.853421 27025 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:21:02.855016 27025 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:21:02.857355 27169 tablet_service.cc:1467] Processing CreateTablet for tablet 44150add33d849faad4b34045ce4164e (DEFAULT_TABLE table=test-table [id=e4440aa076b74a0baabaa0796b0a0f32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:02.857582 27169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44150add33d849faad4b34045ce4164e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:02.859256 27120 tablet_bootstrap.cc:492] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026: Bootstrap starting.
I20260812 14:21:02.859756 27120 tablet_bootstrap.cc:654] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:02.860410 27120 tablet_bootstrap.cc:492] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026: No bootstrap required, opened a new log
I20260812 14:21:02.860466 27120 ts_tablet_manager.cc:1397] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:21:02.860687 27254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.856920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:37689 (local address 127.0.0.1:49350)
0812 14:21:02.856991 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:02.857001 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:02.857037 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:02.857587 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:02.857588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:02.857590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:02.857632 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.857633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.858590 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.858591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.859458 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.859460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.860447 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.860448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.860523 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.860545 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 14:21:02.860597 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 14:21:02.860642 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 14:21:02.860749 27120 raft_consensus.cc:348] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:02.860817 27120 raft_consensus.cc:374] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:02.860864 27120 raft_consensus.cc:729] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60a1fdec533149d686263a94117a1026, State: Initialized, Role: FOLLOWER
I20260812 14:21:02.860905 27120 consensus_queue.cc:260] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:02.861002 27120 ts_tablet_manager.cc:1428] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 14:21:02.861008 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.857217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49350 (local address 127.8.85.3:37689)
0812 14:21:02.857396 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:02.857402 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:02.857413 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:21:02.857455 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:02.857457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:02.857490 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:02.857575 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:02.857700 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.858505 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.859669 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.860384 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.860440 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.860496 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:02.860878 (+ 382us) server_negotiation.cc:299] Negotiation successful
0812 14:21:02.860920 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":111,"threads_started":1}
I20260812 14:21:02.861106 27246 heartbeater.cc:503] Master 127.8.85.62:44863 was elected leader, sending a full tablet report...
I20260812 14:21:02.861505 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.856975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:44133 (local address 127.0.0.1:53096)
0812 14:21:02.857152 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:02.857177 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:02.857231 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:02.857907 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:02.857908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:02.857910 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:02.857947 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.857948 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.859197 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.859198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.860285 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.860288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.861272 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.861273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.861338 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.861342 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:21:02.861400 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:21:02.861439 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":115,"threads_started":1}
I20260812 14:21:02.861544 27568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.857439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53096 (local address 127.8.85.2:44133)
0812 14:21:02.857645 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:02.857649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:02.857659 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:21:02.857699 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:02.857701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:02.857716 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:02.857813 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:02.858057 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.859104 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.860376 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.861197 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.861271 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.861328 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:02.861422 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 14:21:02.861460 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":92,"threads_started":1}
I20260812 14:21:02.861689 27482 tablet_service.cc:1467] Processing CreateTablet for tablet 44150add33d849faad4b34045ce4164e (DEFAULT_TABLE table=test-table [id=e4440aa076b74a0baabaa0796b0a0f32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:02.862069 27482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44150add33d849faad4b34045ce4164e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:02.862223 27331 tablet_service.cc:1467] Processing CreateTablet for tablet 44150add33d849faad4b34045ce4164e (DEFAULT_TABLE table=test-table [id=e4440aa076b74a0baabaa0796b0a0f32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:02.862538 27331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44150add33d849faad4b34045ce4164e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:02.865767 27572 tablet_bootstrap.cc:492] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7: Bootstrap starting.
I20260812 14:21:02.866060 27573 tablet_bootstrap.cc:492] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2: Bootstrap starting.
I20260812 14:21:02.866498 27572 tablet_bootstrap.cc:654] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:02.866767 27573 tablet_bootstrap.cc:654] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:02.867211 27572 log.cc:824] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:02.867398 27573 log.cc:824] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:02.868129 27572 tablet_bootstrap.cc:492] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7: No bootstrap required, opened a new log
I20260812 14:21:02.868183 27573 tablet_bootstrap.cc:492] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2: No bootstrap required, opened a new log
I20260812 14:21:02.868206 27572 ts_tablet_manager.cc:1397] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 14:21:02.868232 27573 ts_tablet_manager.cc:1397] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:21:02.870146 27573 raft_consensus.cc:348] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:02.870146 27572 raft_consensus.cc:348] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:02.870324 27573 raft_consensus.cc:374] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:02.870324 27572 raft_consensus.cc:374] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:02.870362 27573 raft_consensus.cc:729] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2e3ff97abc447b88f6ad61de675ab2, State: Initialized, Role: FOLLOWER
I20260812 14:21:02.870362 27572 raft_consensus.cc:729] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 592082633e164ec1953aca0b5ae9e4f7, State: Initialized, Role: FOLLOWER
I20260812 14:21:02.870496 27573 consensus_queue.cc:260] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:02.870496 27572 consensus_queue.cc:260] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:02.870782 27573 ts_tablet_manager.cc:1428] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:21:02.870782 27572 ts_tablet_manager.cc:1428] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:21:02.870898 27559 heartbeater.cc:503] Master 127.8.85.62:44863 was elected leader, sending a full tablet report...
I20260812 14:21:02.871506 27408 heartbeater.cc:503] Master 127.8.85.62:44863 was elected leader, sending a full tablet report...
I20260812 14:21:02.880590 27576 raft_consensus.cc:482] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:21:02.880666 27576 raft_consensus.cc:504] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:02.880929 27576 leader_election.cc:288] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689), 60a1fdec533149d686263a94117a1026 (127.8.85.1:36371)
I20260812 14:21:02.883893 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.881240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:59585 (local address 127.8.85.3:37689)
0812 14:21:02.881297 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:02.881300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:02.881309 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:21:02.881330 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:02.881331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:02.881342 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:02.881405 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:02.881558 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.882087 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.883105 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.883555 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.883609 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.883617 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:02.883780 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 14:21:02.883817 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:21:02.884025 27255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.881247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:35525 (local address 127.8.85.1:36371)
0812 14:21:02.881461 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:02.881463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:02.881472 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:21:02.881498 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:02.881499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:02.881524 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:02.881588 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:02.881720 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.882233 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.883139 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.883646 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.883716 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.883789 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:02.883897 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 14:21:02.883948 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189}
I20260812 14:21:02.884025 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.881098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:36371 (local address 127.8.85.2:35525)
0812 14:21:02.881356 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:02.881366 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:02.881410 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:02.881591 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:02.881592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:02.881593 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:02.881656 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.881657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.882309 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.882310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.883031 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.883033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.883726 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.883727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.883809 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.883813 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:21:02.883871 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:21:02.883909 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":1,"thread_start_us":90,"threads_started":1}
I20260812 14:21:02.883891 27410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.881083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:37689 (local address 127.8.85.2:59585)
0812 14:21:02.881138 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:02.881151 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:02.881191 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:02.881417 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:02.881419 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:02.881420 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:02.881494 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.881495 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.882163 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.882164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.883021 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:02.883023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.883627 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.883628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:02.883711 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.883715 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:21:02.883768 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:21:02.883811 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 14:21:02.884402 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44150add33d849faad4b34045ce4164e" candidate_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60a1fdec533149d686263a94117a1026" is_pre_election: true
I20260812 14:21:02.884495 27505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44150add33d849faad4b34045ce4164e" candidate_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "592082633e164ec1953aca0b5ae9e4f7" is_pre_election: true
I20260812 14:21:02.884572 27191 raft_consensus.cc:2454] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f2e3ff97abc447b88f6ad61de675ab2 in term 0.
I20260812 14:21:02.884652 27505 raft_consensus.cc:2454] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f2e3ff97abc447b88f6ad61de675ab2 in term 0.
I20260812 14:21:02.884789 27273 leader_election.cc:302] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f2e3ff97abc447b88f6ad61de675ab2, 60a1fdec533149d686263a94117a1026; no voters:
I20260812 14:21:02.884904 27576 raft_consensus.cc:2781] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:21:02.884975 27576 raft_consensus.cc:482] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:21:02.885010 27576 raft_consensus.cc:3037] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:02.885648 27576 raft_consensus.cc:504] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:02.885787 27576 leader_election.cc:288] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [CANDIDATE]: Term 1 election: Requested vote from peers 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689), 60a1fdec533149d686263a94117a1026 (127.8.85.1:36371)
I20260812 14:21:02.885934 27505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44150add33d849faad4b34045ce4164e" candidate_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "592082633e164ec1953aca0b5ae9e4f7"
I20260812 14:21:02.885936 27191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44150add33d849faad4b34045ce4164e" candidate_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60a1fdec533149d686263a94117a1026"
I20260812 14:21:02.886006 27191 raft_consensus.cc:3037] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:02.886006 27505 raft_consensus.cc:3037] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:02.886801 27505 raft_consensus.cc:2454] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f2e3ff97abc447b88f6ad61de675ab2 in term 1.
I20260812 14:21:02.886801 27191 raft_consensus.cc:2454] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f2e3ff97abc447b88f6ad61de675ab2 in term 1.
I20260812 14:21:02.886976 27277 leader_election.cc:302] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f2e3ff97abc447b88f6ad61de675ab2, 592082633e164ec1953aca0b5ae9e4f7; no voters:
I20260812 14:21:02.887096 27576 raft_consensus.cc:2781] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:02.887239 27576 raft_consensus.cc:686] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [term 1 LEADER]: Becoming Leader. State: Replica: 2f2e3ff97abc447b88f6ad61de675ab2, State: Running, Role: LEADER
I20260812 14:21:02.887338 27576 consensus_queue.cc:237] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:02.888171 27025 catalog_manager.cc:5495] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2). New cstate: current_term: 1 leader_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } health_report { overall_health: UNKNOWN } } }
I20260812 14:21:02.899508 27568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.897219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53100 (local address 127.8.85.2:44133)
0812 14:21:02.897275 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:02.897278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:02.897287 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:21:02.897375 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:02.897376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:02.897392 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:02.897459 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:21:02.897599 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.898036 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.898739 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:02.898921 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:02.899017 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:02.899180 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:21:02.899291 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:21:02.899365 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:02.899405 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 14:21:02.899438 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:21:02.901122 27505 raft_consensus.cc:1261] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7 [term 1 FOLLOWER]: Refusing update from remote peer 2f2e3ff97abc447b88f6ad61de675ab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:21:02.901122 27191 raft_consensus.cc:1261] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Refusing update from remote peer 2f2e3ff97abc447b88f6ad61de675ab2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:21:02.901512 27576 consensus_queue.cc:1035] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:21:02.901666 27579 consensus_queue.cc:1035] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:21:02.903451 27583 mvcc.cc:205] Tried to move back new op lower bound from 7317686120040550400 to 7317686119987437568. Current Snapshot: MvccSnapshot[applied={T|T < 7317686120040550400}]
I20260812 14:21:02.903636 27585 mvcc.cc:205] Tried to move back new op lower bound from 7317686120040550400 to 7317686119987437568. Current Snapshot: MvccSnapshot[applied={T|T < 7317686120040550400}]
I20260812 14:21:02.904666 27587 mvcc.cc:205] Tried to move back new op lower bound from 7317686120040550400 to 7317686119987437568. Current Snapshot: MvccSnapshot[applied={T|T < 7317686120040550400}]
I20260812 14:21:02.906167 27024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:21:02.906255 27024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:21:02.906947 27024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Sending DeleteTablet for 3 replicas of tablet 44150add33d849faad4b34045ce4164e
I20260812 14:21:02.907240 27482 tablet_service.cc:1514] Processing DeleteTablet for tablet 44150add33d849faad4b34045ce4164e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:02 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:49350
I20260812 14:21:02.907266 27331 tablet_service.cc:1514] Processing DeleteTablet for tablet 44150add33d849faad4b34045ce4164e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:02 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:53096
I20260812 14:21:02.907298 27169 tablet_service.cc:1514] Processing DeleteTablet for tablet 44150add33d849faad4b34045ce4164e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:02 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:45808
I20260812 14:21:02.907389 27256 tablet_replica.cc:330] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026: stopping tablet replica
I20260812 14:21:02.907461 27256 raft_consensus.cc:2229] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:21:02.907516 27256 raft_consensus.cc:2258] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:02.907555 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 27072
I20260812 14:21:02.911273 27596 tablet_replica.cc:330] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2: stopping tablet replica
I20260812 14:21:02.911388 27596 raft_consensus.cc:2229] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:21:02.911520 27596 raft_consensus.cc:2258] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:02.911666 27595 tablet_replica.cc:330] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7: stopping tablet replica
I20260812 14:21:02.911758 27595 raft_consensus.cc:2229] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:21:02.911863 27595 raft_consensus.cc:2258] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:02.912119 27596 ts_tablet_manager.cc:1905] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:21:02.912284 27595 ts_tablet_manager.cc:1905] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:21:02.914175 27596 ts_tablet_manager.cc:1918] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:21:02.914309 27596 log.cc:1196] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/44150add33d849faad4b34045ce4164e
I20260812 14:21:02.914362 27595 ts_tablet_manager.cc:1918] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:21:02.914448 27595 log.cc:1196] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/44150add33d849faad4b34045ce4164e
I20260812 14:21:02.914739 27595 ts_tablet_manager.cc:1939] T 44150add33d849faad4b34045ce4164e P 592082633e164ec1953aca0b5ae9e4f7: Deleting consensus metadata
I20260812 14:21:02.914916 27596 ts_tablet_manager.cc:1939] T 44150add33d849faad4b34045ce4164e P 2f2e3ff97abc447b88f6ad61de675ab2: Deleting consensus metadata
I20260812 14:21:02.915365 27009 catalog_manager.cc:4841] TS 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689): tablet 44150add33d849faad4b34045ce4164e (table test-table [id=e4440aa076b74a0baabaa0796b0a0f32]) successfully deleted
I20260812 14:21:02.915565 27008 catalog_manager.cc:4841] TS 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133): tablet 44150add33d849faad4b34045ce4164e (table test-table [id=e4440aa076b74a0baabaa0796b0a0f32]) successfully deleted
W20260812 14:21:02.917929 27007 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:36371 (error 108)
I20260812 14:21:02.918471 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:02.918321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:36371 (local address 127.0.0.1:45822)
0812 14:21:02.918383 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:02.918430 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:36371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 14:21:02.918550 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 27253
W20260812 14:21:02.918723 27007 catalog_manager.cc:4584] TS 60a1fdec533149d686263a94117a1026 (127.8.85.1:36371): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 44150add33d849faad4b34045ce4164e: Network error: Client connection negotiation failed: client connection to 127.8.85.1:36371: connect: Connection refused (error 111)
I20260812 14:21:02.928076 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 27411
I20260812 14:21:02.934644 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 26987
I20260812 14:21:22.942169 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:44863
--webserver_interface=127.8.85.62
--webserver_port=41543
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:44863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:23.025939 27599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:23.026207 27599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:23.026243 27599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:23.027384 27599 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 14:21:23.027493 27599 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 14:21:23.027529 27599 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 14:21:23.027551 27599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:23.027582 27599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:23.027612 27599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:23.027644 27599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:23.027673 27599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:23.029174 27599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:44863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:44863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=41543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:23.029458 27599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:23.029693 27599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:21:23.034782 27599 server_base.cc:1034] running on GCE node
W20260812 14:21:23.034906 27610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:23.034858 27607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:23.034868 27608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:23.035328 27599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:23.035669 27599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:23.036825 27599 hybrid_clock.cc:648] HybridClock initialized: now 1786544483036814 us; error 30 us; skew 500 ppm
Aug 12 14:21:23 dist-test-slave-5k54 krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544483, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:23.044322 27599 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:21:23.044821 27599 webserver.cc:460] Webserver started at http://127.8.85.62:41543/ using document root <none> and password file <none>
I20260812 14:21:23.045053 27599 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:23.045123 27599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:23.046701 27599 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:21:23.047708 27618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:23.047932 27599 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:21:23.048039 27599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "ed7556c2c93a4517ad510de1ef06e3d0"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "a990f63ccf576d77d9aa6300525d6dc6"
server_key_iv: "c6777239f8b9cef48c52bcbcd5ba55e5"
server_key_version: "encryptionkey@0"
I20260812 14:21:23.048409 27599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:23.089449 27599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:23.092067 27599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:23.092247 27599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:23.096196 27599 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:44863
I20260812 14:21:23.096256 27677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:44863 every 8 connection(s)
I20260812 14:21:23.096660 27599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:23.098161 27678 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:21:23.098662 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 27599
I20260812 14:21:23.099007 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:36371
--local_ip_for_outbound_sockets=127.8.85.1
--tserver_master_addrs=127.8.85.62:44863
--webserver_port=36299
--webserver_interface=127.8.85.1
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:21:23.099094 27678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Bootstrap starting.
I20260812 14:21:23.108338 27678 log.cc:824] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:23.110706 27678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:21:23.111070 27678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Bootstrap complete.
I20260812 14:21:23.113399 27678 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } }
I20260812 14:21:23.113665 27678 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed7556c2c93a4517ad510de1ef06e3d0, State: Initialized, Role: FOLLOWER
I20260812 14:21:23.113840 27678 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } }
I20260812 14:21:23.113910 27678 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:23.113974 27678 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:23.114023 27678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 2 FOLLOWER]: Advancing to term 3
I20260812 14:21:23.115046 27678 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } }
I20260812 14:21:23.115140 27678 leader_election.cc:302] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed7556c2c93a4517ad510de1ef06e3d0; no voters:
I20260812 14:21:23.115363 27678 leader_election.cc:288] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 14:21:23.115407 27683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 14:21:23.115563 27683 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [term 3 LEADER]: Becoming Leader. State: Replica: ed7556c2c93a4517ad510de1ef06e3d0, State: Running, Role: LEADER
I20260812 14:21:23.115662 27678 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:23.115653 27683 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } }
I20260812 14:21:23.116000 27685 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed7556c2c93a4517ad510de1ef06e3d0. Latest consensus state: current_term: 3 leader_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } } }
I20260812 14:21:23.116103 27685 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:23.115996 27684 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7556c2c93a4517ad510de1ef06e3d0" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 44863 } } }
I20260812 14:21:23.116331 27689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:23.116364 27684 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:23.117224 27689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:23.117336 27689 catalog_manager.cc:1259] Loaded cluster ID: 99918eb5073c464ba87dac70abaaef65
I20260812 14:21:23.117373 27689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:21:23.118623 27689 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:23.118810 27689 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Loaded TSK: 0
I20260812 14:21:23.118863 27689 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ed7556c2c93a4517ad510de1ef06e3d0: Loaded TSK: 1
F20260812 14:21:23.119083 27689 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fec246ec9ad google::LogMessage::Fail() at ??:0
@ 0x7fec246ef88a google::LogMessage::SendToLog() at ??:0
@ 0x7fec246ec4de google::LogMessage::Flush() at ??:0
@ 0x7fec246eda19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fec26e59104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fec26e56d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fec2c64b1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fec2c64b48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fec2c64b594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fec2c611c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fec2c61bad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fec265ba0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fec265af954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fec274136db start_thread at ??:0
@ 0x7fec231ea61f clone at ??:0
W20260812 14:21:23.212348 27681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:23.212589 27681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:23.212626 27681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:23.213779 27681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:23.213847 27681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:23.213893 27681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:21:23.215363 27681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:36371
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=36299
--tserver_master_addrs=127.8.85.62:44863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:23.215648 27681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:23.215883 27681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:23.220867 27728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:23.220916 27731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:23.220932 27729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:23.221159 27681 server_base.cc:1034] running on GCE node
I20260812 14:21:23.221426 27681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:23.221714 27681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:23.222906 27681 hybrid_clock.cc:648] HybridClock initialized: now 1786544483222886 us; error 34 us; skew 500 ppm
Aug 12 14:21:23 dist-test-slave-5k54 krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544483, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:23.225900 27681 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:21:23.226380 27681 webserver.cc:460] Webserver started at http://127.8.85.1:36299/ using document root <none> and password file <none>
I20260812 14:21:23.226646 27681 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:23.226718 27681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:23.228188 27681 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:21:23.228943 27739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:23.229151 27681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:21:23.229244 27681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "60a1fdec533149d686263a94117a1026"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "28f6d0485e0883e3b284ac757bb901c9"
server_key_iv: "39a63e93bedf06d41c191889f43202ce"
server_key_version: "encryptionkey@0"
I20260812 14:21:23.229602 27681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:23.240343 27681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:23.242614 27681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:23.242774 27681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:23.243010 27681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:23.243494 27746 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 14:21:23.245504 27681 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 14:21:23.245564 27681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 14:21:23.245609 27681 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 14:21:23.246227 27681 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 14:21:23.246282 27681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 14:21:23.246369 27746 tablet_bootstrap.cc:492] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026: Bootstrap starting.
I20260812 14:21:23.253630 27681 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:36371
I20260812 14:21:23.253752 27878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:36371 every 8 connection(s)
I20260812 14:21:23.254061 27681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:21:23.255592 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 27681
I20260812 14:21:23.255903 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:44133
--local_ip_for_outbound_sockets=127.8.85.2
--tserver_master_addrs=127.8.85.62:44863
--webserver_port=37311
--webserver_interface=127.8.85.2
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:21:23.257361 27746 log.cc:824] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:23.260097 27746 tablet_bootstrap.cc:492] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:21:23.260434 27746 tablet_bootstrap.cc:492] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026: Bootstrap complete.
I20260812 14:21:23.261592 27746 ts_tablet_manager.cc:1397] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.011s
I20260812 14:21:23.263103 27746 raft_consensus.cc:348] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:23.263474 27746 raft_consensus.cc:729] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60a1fdec533149d686263a94117a1026, State: Initialized, Role: FOLLOWER
I20260812 14:21:23.263652 27746 consensus_queue.cc:260] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:23.263959 27746 ts_tablet_manager.cc:1428] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
W20260812 14:21:23.336668 27882 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:21:23.254761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:54101)
0812 14:21:23.254940 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:23.254987 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:23.255566 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:23.336446 (+ 80880us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
W20260812 14:21:23.336958 27880 heartbeater.cc:644] Failed to heartbeat to 127.8.85.62:44863 (0 consecutive failures): Network error: Failed to ping master at 127.8.85.62:44863: Client connection negotiation failed: client connection to 127.8.85.62:44863: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 14:21:23.337328 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:23.337186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:45195)
0812 14:21:23.337249 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:23.337301 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 14:21:23.337675 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:23.337600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:45215)
0812 14:21:23.337628 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:23.337651 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260812 14:21:23.337817 27880 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:21:23.342015 27884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:23.342198 27884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:23.342249 27884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:23.343398 27884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:23.343498 27884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:23.343552 27884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:21:23.345058 27884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:44133
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=37311
--tserver_master_addrs=127.8.85.62:44863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:23.345311 27884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:23.345544 27884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:23.350513 27892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:23.350539 27893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:23.350659 27884 server_base.cc:1034] running on GCE node
W20260812 14:21:23.350775 27895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:23.350991 27884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:23.351197 27884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:23.352337 27884 hybrid_clock.cc:648] HybridClock initialized: now 1786544483352317 us; error 38 us; skew 500 ppm
Aug 12 14:21:23 dist-test-slave-5k54 krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544483, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:23.354777 27884 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:21:23.355245 27884 webserver.cc:460] Webserver started at http://127.8.85.2:37311/ using document root <none> and password file <none>
I20260812 14:21:23.355659 27884 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:23.355712 27884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:23.357003 27884 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:21:23.357594 27903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:23.357787 27884 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:21:23.357872 27884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "2f2e3ff97abc447b88f6ad61de675ab2"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "c36f77d56668fe1032534fa07c897f9c"
server_key_iv: "791242753dc8936d7558007f983f046b"
server_key_version: "encryptionkey@0"
I20260812 14:21:23.358187 27884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:23.369824 27884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:23.371620 27884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:23.371742 27884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:23.371907 27884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:23.372154 27884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:23.372185 27884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:23.372205 27884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:23.372218 27884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:23.378007 27884 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:44133
I20260812 14:21:23.378083 28033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:44133 every 8 connection(s)
I20260812 14:21:23.378436 27884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:21:23.379577 28036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:23.379149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:46513)
0812 14:21:23.379302 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:23.379370 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
W20260812 14:21:23.380023 28034 heartbeater.cc:644] Failed to heartbeat to 127.8.85.62:44863 (0 consecutive failures): Network error: Failed to ping master at 127.8.85.62:44863: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
I20260812 14:21:23.380383 28036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:23.380230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:45445)
0812 14:21:23.380328 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:23.380361 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 14:21:23.380762 28036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:23.380660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:46353)
0812 14:21:23.380716 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:23.380743 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 14:21:23.380924 28034 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 14:21:23.381834 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 27884
I20260812 14:21:23.382134 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:37689
--local_ip_for_outbound_sockets=127.8.85.3
--tserver_master_addrs=127.8.85.62:44863
--webserver_port=44017
--webserver_interface=127.8.85.3
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:23.465762 28037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:23.466005 28037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:23.466048 28037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:23.467224 28037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:23.467295 28037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:23.467341 28037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:21:23.468858 28037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:37689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=44017
--tserver_master_addrs=127.8.85.62:44863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:23.469154 28037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:23.469399 28037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:23.474403 28045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:23.474380 28048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:23.474427 28037 server_base.cc:1034] running on GCE node
W20260812 14:21:23.474371 28046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:23.474753 28037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:23.475014 28037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:23.476161 28037 hybrid_clock.cc:648] HybridClock initialized: now 1786544483476142 us; error 35 us; skew 500 ppm
Aug 12 14:21:23 dist-test-slave-5k54 krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544483, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:23.478695 28037 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:21:23.479115 28037 webserver.cc:460] Webserver started at http://127.8.85.3:44017/ using document root <none> and password file <none>
I20260812 14:21:23.479333 28037 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:23.479385 28037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:23.480783 28037 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:21:23.481375 28055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:23.481559 28037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:21:23.481666 28037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "592082633e164ec1953aca0b5ae9e4f7"
format_stamp: "Formatted at 2026-08-12 14:20:41 on dist-test-slave-5k54"
server_key: "69f63708da72b64145bdedc26850002e"
server_key_iv: "f4d126f611264a7ba1a03ffc770de062"
server_key_version: "encryptionkey@0"
I20260812 14:21:23.482007 28037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:23.492666 28037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:23.494398 28037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:23.494555 28037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:23.494771 28037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:23.495069 28037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:23.495124 28037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:23.495163 28037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:23.495193 28037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:23.501062 28037 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:37689
I20260812 14:21:23.501111 28188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:37689 every 8 connection(s)
I20260812 14:21:23.501439 28037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:21:23.502396 28191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:23.502041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:34177)
0812 14:21:23.502191 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:23.502258 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
W20260812 14:21:23.502808 28189 heartbeater.cc:644] Failed to heartbeat to 127.8.85.62:44863 (0 consecutive failures): Network error: Failed to ping master at 127.8.85.62:44863: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
I20260812 14:21:23.503122 28191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:23.503033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:51677)
0812 14:21:23.503078 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:23.503105 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 14:21:23.503415 28191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:23.503344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:49975)
0812 14:21:23.503384 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:23.503403 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 14:21:23.503590 28189 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 14:21:23.507843 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 28037
I20260812 14:21:24.338368 28194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:24.338138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:47173)
0812 14:21:24.338269 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:24.338300 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260812 14:21:24.381456 28195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:24.381240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:47973)
0812 14:21:24.381357 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:24.381406 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260812 14:21:24.504127 28196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:24.503909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:35821)
0812 14:21:24.504034 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:24.504074 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260812 14:21:24.612617 28197 raft_consensus.cc:482] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:21:24.612747 28197 raft_consensus.cc:504] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:24.613149 28197 leader_election.cc:288] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689), 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133)
Aug 12 14:21:24 dist-test-slave-5k54 krb5kdc[26340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544483, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.2@KRBTEST.COM
Aug 12 14:21:24 dist-test-slave-5k54 krb5kdc[26340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544483, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.3@KRBTEST.COM
I20260812 14:21:24.624581 28198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:24.613373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:44133 (local address 127.8.85.1:52713)
0812 14:21:24.613513 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:24.613524 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:24.613580 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:24.614422 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:24.614423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:24.614835 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:24.615027 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:24.615034 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:24.616271 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:24.616273 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:24.617047 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:24.617049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:24.617698 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:24.617699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:24.617797 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:24.618435 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:21:24.618439 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:24.620180 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:24.622744 (+ 2564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:24.622750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:24.622752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:24.622986 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:24.623327 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:24.623328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:24.623328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:24.623493 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:24.624017 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:24.624024 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:24.624139 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 14:21:24.624231 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 14:21:24.624467 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":9,"thread_start_us":79,"threads_started":1}
I20260812 14:21:24.624614 28199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:24.613462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:52713 (local address 127.8.85.2:44133)
0812 14:21:24.613616 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:24.613630 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:24.613676 (+ 46us) server_negotiation.cc:407] Connection header received
0812 14:21:24.614177 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:24.614180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:24.614223 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:24.614394 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:24.615160 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:24.616160 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:24.617146 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:24.617588 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:24.617734 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:24.620320 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:24.620324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:24.620325 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:24.620336 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:24.622639 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:24.623087 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:24.623091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:24.623092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:24.623221 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:24.623566 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:24.623567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:24.623567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:24.623919 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:24.624011 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:24.624343 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 14:21:24.624496 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260812 14:21:24.624688 28194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:24.613341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:37689 (local address 127.8.85.1:33717)
0812 14:21:24.613402 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:24.613415 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:24.613476 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:24.614418 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:24.614422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:24.614834 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:24.615028 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:24.615034 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:24.616201 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:24.616202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:24.617048 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:24.617051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:24.617669 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:24.617671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:24.617801 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:24.618392 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:21:24.618402 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:24.620714 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:24.623204 (+ 2490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:24.623204 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:24.623205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:24.623477 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:24.623764 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:24.623764 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:24.623765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:24.623837 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:24.624305 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:24.624306 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:24.624361 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 14:21:24.624466 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 14:21:24.624561 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":60}
I20260812 14:21:24.624864 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:24.613500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:33717 (local address 127.8.85.3:37689)
0812 14:21:24.613677 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:24.613694 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:24.613723 (+ 29us) server_negotiation.cc:407] Connection header received
0812 14:21:24.614174 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:24.614176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:24.614222 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:24.614367 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:24.615167 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:24.616094 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:24.617172 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:24.617567 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:24.617680 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:24.620837 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:24.620841 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:24.620841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:24.620851 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:24.623114 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:24.623575 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:24.623578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:24.623579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:24.623686 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:24.623906 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:24.623907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:24.623908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:24.624194 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:24.624304 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:24.624549 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 14:21:24.624695 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
W20260812 14:21:24.625698 27742 leader_election.cc:341] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689): Not found: Tablet not found: 44150add33d849faad4b34045ce4164e
W20260812 14:21:24.625892 27741 leader_election.cc:341] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133): Not found: Tablet not found: 44150add33d849faad4b34045ce4164e
I20260812 14:21:24.625944 27741 leader_election.cc:302] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60a1fdec533149d686263a94117a1026; no voters: 2f2e3ff97abc447b88f6ad61de675ab2, 592082633e164ec1953aca0b5ae9e4f7
I20260812 14:21:24.626116 28197 raft_consensus.cc:2726] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:21:25.339038 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:25.338812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:46509)
0812 14:21:25.338949 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:25.338989 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260812 14:21:25.382122 28203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:25.381920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:43875)
0812 14:21:25.382041 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:25.382078 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
I20260812 14:21:25.504911 28204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:25.504644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:36347)
0812 14:21:25.504814 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:25.504866 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":68,"threads_started":1}
I20260812 14:21:26.161679 28205 raft_consensus.cc:482] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:21:26.161813 28205 raft_consensus.cc:504] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:26.162031 28205 leader_election.cc:288] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689), 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133)
W20260812 14:21:26.162461 27741 leader_election.cc:341] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133): Not found: Tablet not found: 44150add33d849faad4b34045ce4164e
W20260812 14:21:26.162563 27742 leader_election.cc:341] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689): Not found: Tablet not found: 44150add33d849faad4b34045ce4164e
I20260812 14:21:26.162674 27742 leader_election.cc:302] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60a1fdec533149d686263a94117a1026; no voters: 2f2e3ff97abc447b88f6ad61de675ab2, 592082633e164ec1953aca0b5ae9e4f7
I20260812 14:21:26.162746 28205 raft_consensus.cc:2726] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:21:26.339802 28206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:26.339592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:35539)
0812 14:21:26.339714 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:26.339752 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260812 14:21:26.382748 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:26.382552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:55245)
0812 14:21:26.382674 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:26.382710 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260812 14:21:26.505722 28208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:26.505467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:42695)
0812 14:21:26.505623 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:26.505669 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":83,"threads_started":1}
I20260812 14:21:27.340535 28209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:27.340307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:52675)
0812 14:21:27.340448 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:27.340488 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260812 14:21:27.383599 28210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:27.383320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:53409)
0812 14:21:27.383523 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:27.383558 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":62,"threads_started":1}
I20260812 14:21:27.506393 28211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:27.506183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:60565)
0812 14:21:27.506315 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:27.506350 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260812 14:21:28.341338 28214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:28.341106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:47227)
0812 14:21:28.341246 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:28.341289 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":62,"threads_started":1}
I20260812 14:21:28.384204 28215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:28.384014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:33961)
0812 14:21:28.384129 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:28.384164 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260812 14:21:28.507164 28216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:28.506954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:48411)
0812 14:21:28.507088 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:28.507123 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260812 14:21:29.154520 28217 raft_consensus.cc:482] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:21:29.154613 28217 raft_consensus.cc:504] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:29.154837 28217 leader_election.cc:288] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689), 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133)
W20260812 14:21:29.155238 27741 leader_election.cc:341] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133): Not found: Tablet not found: 44150add33d849faad4b34045ce4164e
W20260812 14:21:29.155310 27742 leader_election.cc:341] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689): Not found: Tablet not found: 44150add33d849faad4b34045ce4164e
I20260812 14:21:29.155339 27741 leader_election.cc:302] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60a1fdec533149d686263a94117a1026; no voters: 2f2e3ff97abc447b88f6ad61de675ab2, 592082633e164ec1953aca0b5ae9e4f7
I20260812 14:21:29.155426 28217 raft_consensus.cc:2726] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:21:29.342190 28219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:29.341907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:60205)
0812 14:21:29.342076 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:29.342120 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":67,"threads_started":1}
I20260812 14:21:29.384963 28220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:29.384641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:55849)
0812 14:21:29.384876 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:29.384917 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":75,"threads_started":1}
I20260812 14:21:29.507932 28221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:29.507705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:56815)
0812 14:21:29.507844 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:29.507888 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260812 14:21:30.342989 28222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:30.342768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:43069)
0812 14:21:30.342902 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:30.342941 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260812 14:21:30.385703 28223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:30.385500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:46509)
0812 14:21:30.385621 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:30.385659 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260812 14:21:30.508594 28224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:30.508376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:44627)
0812 14:21:30.508509 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:30.508549 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260812 14:21:31.343735 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:31.343524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:46467)
0812 14:21:31.343650 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:31.343689 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260812 14:21:31.386293 28227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:31.386096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:50509)
0812 14:21:31.386217 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:31.386250 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260812 14:21:31.509295 28228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:31.509056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:45739)
0812 14:21:31.509214 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:31.509248 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260812 14:21:32.344463 28229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:32.344194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:35367)
0812 14:21:32.344360 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:32.344416 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":74,"threads_started":1}
I20260812 14:21:32.386915 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:32.386701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:46413)
0812 14:21:32.386837 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:32.386872 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260812 14:21:32.510028 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:32.509763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:58653)
0812 14:21:32.509931 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:32.509972 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
I20260812 14:21:32.886066 28233 raft_consensus.cc:482] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:21:32.886157 28233 raft_consensus.cc:504] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:32.886318 28233 leader_election.cc:288] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689), 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133)
W20260812 14:21:32.886720 27742 leader_election.cc:341] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689): Not found: Tablet not found: 44150add33d849faad4b34045ce4164e
W20260812 14:21:32.886795 27741 leader_election.cc:341] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133): Not found: Tablet not found: 44150add33d849faad4b34045ce4164e
I20260812 14:21:32.886873 27741 leader_election.cc:302] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60a1fdec533149d686263a94117a1026; no voters: 2f2e3ff97abc447b88f6ad61de675ab2, 592082633e164ec1953aca0b5ae9e4f7
I20260812 14:21:32.886976 28233 raft_consensus.cc:2726] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:21:33.345232 28234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:33.345005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:37245)
0812 14:21:33.345143 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:33.345182 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
I20260812 14:21:33.387612 28235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:33.387374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:38553)
0812 14:21:33.387531 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:33.387569 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":110,"threads_started":1}
I20260812 14:21:33.510778 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:33.510526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:36439)
0812 14:21:33.510691 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:33.510729 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":80,"threads_started":1}
I20260812 14:21:34.346031 28237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:34.345768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:53861)
0812 14:21:34.345942 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:34.345976 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
I20260812 14:21:34.388219 28238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:34.388026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:45451)
0812 14:21:34.388142 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:34.388175 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260812 14:21:34.511605 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:34.511345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:38549)
0812 14:21:34.511518 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:34.511557 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":119,"threads_started":1}
I20260812 14:21:35.048229 28241 raft_consensus.cc:482] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:21:35.048327 28241 raft_consensus.cc:504] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "592082633e164ec1953aca0b5ae9e4f7" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37689 } } peers { permanent_uuid: "2f2e3ff97abc447b88f6ad61de675ab2" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44133 } } peers { permanent_uuid: "60a1fdec533149d686263a94117a1026" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 36371 } }
I20260812 14:21:35.048561 28241 leader_election.cc:288] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689), 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133)
W20260812 14:21:35.048954 27742 leader_election.cc:341] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 592082633e164ec1953aca0b5ae9e4f7 (127.8.85.3:37689): Not found: Tablet not found: 44150add33d849faad4b34045ce4164e
W20260812 14:21:35.049081 27741 leader_election.cc:341] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f2e3ff97abc447b88f6ad61de675ab2 (127.8.85.2:44133): Not found: Tablet not found: 44150add33d849faad4b34045ce4164e
I20260812 14:21:35.049144 27741 leader_election.cc:302] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60a1fdec533149d686263a94117a1026; no voters: 2f2e3ff97abc447b88f6ad61de675ab2, 592082633e164ec1953aca0b5ae9e4f7
I20260812 14:21:35.049224 28241 raft_consensus.cc:2726] T 44150add33d849faad4b34045ce4164e P 60a1fdec533149d686263a94117a1026 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:21:35.346722 28242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:35.346484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:43913)
0812 14:21:35.346614 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:35.346653 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260812 14:21:35.388818 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:35.388619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:59249)
0812 14:21:35.388740 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:35.388778 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":56,"threads_started":1}
I20260812 14:21:35.512305 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:35.512068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:36263)
0812 14:21:35.512221 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:35.512259 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260812 14:21:36.347489 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:36.347213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:49905)
0812 14:21:36.347366 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:36.347414 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
I20260812 14:21:36.389428 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:36.389243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:39303)
0812 14:21:36.389355 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:36.389389 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260812 14:21:36.513043 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:36.512833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:47947)
0812 14:21:36.512959 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:36.512997 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260812 14:21:37.348214 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:37.347997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:55583)
0812 14:21:37.348125 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:37.348166 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":67,"threads_started":1}
I20260812 14:21:37.390077 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:37.389854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:46271)
0812 14:21:37.389999 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:37.390033 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260812 14:21:37.513772 28252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:37.513545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:53581)
0812 14:21:37.513682 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:37.513723 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
I20260812 14:21:38.348984 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:38.348738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.1:50065)
0812 14:21:38.348879 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:38.348935 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":60,"threads_started":1}
I20260812 14:21:38.390904 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:38.390664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.2:60139)
0812 14:21:38.390825 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:38.390860 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":60,"threads_started":1}
I20260812 14:21:38.510219 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 27681
I20260812 14:21:38.516458 28256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:38.515312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:44863 (local address 127.8.85.3:55283)
0812 14:21:38.516362 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:38.516399 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.62:44863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1031,"thread_start_us":983,"threads_started":1}
I20260812 14:21:38.522409 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 27884
I20260812 14:21:38.527976 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 28037
2026-08-12T14:21:38Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58567 ms)
[----------] 30 tests from SecurityITest (235018 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:21:38 dist-test-slave-5k54 krb5kdc[28259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:21:38 dist-test-slave-5k54 krb5kdc[28259](info): set up 2 sockets
Aug 12 14:21:38 dist-test-slave-5k54 krb5kdc[28259](info): commencing operation
krb5kdc: starting...
W20260812 14:21:39.654624 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:21:39 dist-test-slave-5k54 krb5kdc[28259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:21:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:21:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:21:39.777967 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:36807
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:36807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:39.868845 28277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:39.869132 28277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:39.869192 28277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:39.870352 28277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:39.870389 28277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:39.870404 28277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:39.870416 28277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:39.870428 28277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:39.871873 28277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:36807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:36807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:39.872063 28277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:39.872233 28277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:39.877178 28285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:39.877130 28283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:39.877291 28282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:39.877355 28277 server_base.cc:1034] running on GCE node
I20260812 14:21:39.877665 28277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:39.878003 28277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:39.879174 28277 hybrid_clock.cc:648] HybridClock initialized: now 1786544499879170 us; error 38 us; skew 500 ppm
Aug 12 14:21:39 dist-test-slave-5k54 krb5kdc[28259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544499, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:39.887538 28277 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:21:39.888003 28277 webserver.cc:460] Webserver started at http://127.8.85.62:46151/ using document root <none> and password file <none>
I20260812 14:21:39.888255 28277 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:39.888324 28277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:39.888475 28277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:39.889533 28277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "b73bf1f9df5b44aabf4635cc7142ef45"
format_stamp: "Formatted at 2026-08-12 14:21:39 on dist-test-slave-5k54"
server_key: "6129313eee0f4048a68a0f29fcb2279f"
server_key_iv: "61cf17a84e43e9fa98c59d2b2a6750ab"
server_key_version: "encryptionkey@0"
I20260812 14:21:39.889901 28277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "b73bf1f9df5b44aabf4635cc7142ef45"
format_stamp: "Formatted at 2026-08-12 14:21:39 on dist-test-slave-5k54"
server_key: "6129313eee0f4048a68a0f29fcb2279f"
server_key_iv: "61cf17a84e43e9fa98c59d2b2a6750ab"
server_key_version: "encryptionkey@0"
I20260812 14:21:39.891269 28277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:21:39.892081 28295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:39.892253 28277 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:21:39.892361 28277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "b73bf1f9df5b44aabf4635cc7142ef45"
format_stamp: "Formatted at 2026-08-12 14:21:39 on dist-test-slave-5k54"
server_key: "6129313eee0f4048a68a0f29fcb2279f"
server_key_iv: "61cf17a84e43e9fa98c59d2b2a6750ab"
server_key_version: "encryptionkey@0"
I20260812 14:21:39.892438 28277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:39.903267 28277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:39.905573 28277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:39.905735 28277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:39.909910 28277 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:36807
I20260812 14:21:39.909955 28358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:36807 every 8 connection(s)
I20260812 14:21:39.910332 28277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:39.911286 28359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:39.913580 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 28277
I20260812 14:21:39.913607 28359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45: Bootstrap starting.
I20260812 14:21:39.913699 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:21:39.913895 8532 external_mini_cluster.cc:1428] Setting key 4b031b14c4256a628ca02503d6980db5
I20260812 14:21:39.914335 28359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:39.914700 28359 log.cc:824] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:39.915562 28359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45: No bootstrap required, opened a new log
I20260812 14:21:39.917582 28359 raft_consensus.cc:348] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73bf1f9df5b44aabf4635cc7142ef45" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 36807 } }
I20260812 14:21:39.917719 28359 raft_consensus.cc:374] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:39.917742 28359 raft_consensus.cc:729] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b73bf1f9df5b44aabf4635cc7142ef45, State: Initialized, Role: FOLLOWER
I20260812 14:21:39.917877 28359 consensus_queue.cc:260] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73bf1f9df5b44aabf4635cc7142ef45" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 36807 } }
I20260812 14:21:39.917954 28359 raft_consensus.cc:388] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:39.918009 28359 raft_consensus.cc:482] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:39.918057 28359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:39.918620 28359 raft_consensus.cc:504] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73bf1f9df5b44aabf4635cc7142ef45" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 36807 } }
I20260812 14:21:39.918744 28359 leader_election.cc:302] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b73bf1f9df5b44aabf4635cc7142ef45; no voters:
I20260812 14:21:39.918926 28359 leader_election.cc:288] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:21:39.918967 28364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:39.919132 28364 raft_consensus.cc:686] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [term 1 LEADER]: Becoming Leader. State: Replica: b73bf1f9df5b44aabf4635cc7142ef45, State: Running, Role: LEADER
I20260812 14:21:39.919199 28359 sys_catalog.cc:564] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:39.919279 28364 consensus_queue.cc:237] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73bf1f9df5b44aabf4635cc7142ef45" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 36807 } }
I20260812 14:21:39.919663 28364 sys_catalog.cc:455] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b73bf1f9df5b44aabf4635cc7142ef45. Latest consensus state: current_term: 1 leader_uuid: "b73bf1f9df5b44aabf4635cc7142ef45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73bf1f9df5b44aabf4635cc7142ef45" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 36807 } } }
I20260812 14:21:39.919675 28366 sys_catalog.cc:455] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b73bf1f9df5b44aabf4635cc7142ef45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b73bf1f9df5b44aabf4635cc7142ef45" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 36807 } } }
I20260812 14:21:39.919858 28364 sys_catalog.cc:458] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:39.919872 28366 sys_catalog.cc:458] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:39.920109 28373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:39.920655 28373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 14:21:39 dist-test-slave-5k54 krb5kdc[28259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:21:39.924182 28373 catalog_manager.cc:1347] Generated new cluster ID: 42893755571841a5a8699f7b1044b8f1
I20260812 14:21:39.924240 28373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:21:39.925218 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:39.914443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40058 (local address 127.8.85.62:36807)
0812 14:21:39.914629 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:39.914636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:39.914661 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:21:39.915295 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:39.915302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:39.915371 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:39.915610 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:39.916266 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:39.917278 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:39.918275 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:39.918678 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:39.918820 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:39.921774 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:39.921779 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:39.921779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:39.921793 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:39.923564 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:39.924004 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:39.924008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:39.924008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:39.924136 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:39.924346 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:39.924347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:39.924347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:39.924577 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:39.924740 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:39.924856 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 14:21:39.925005 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260812 14:21:39.939455 28373 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:21:39.939980 28373 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:39.944404 28373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45: Generated new TSK 0
I20260812 14:21:39.944576 28373 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:21:39.970999 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:36807
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:40.054417 28389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:40.054651 28389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:40.054692 28389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:40.055934 28389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:40.055999 28389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:40.056047 28389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:21:40.057503 28389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:36807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:40.057765 28389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:40.058003 28389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:40.062937 28394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:40.062906 28399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:40.062944 28395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:40.063146 28389 server_base.cc:1034] running on GCE node
I20260812 14:21:40.063475 28389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:40.063741 28389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:40.064903 28389 hybrid_clock.cc:648] HybridClock initialized: now 1786544500064923 us; error 62 us; skew 500 ppm
Aug 12 14:21:40 dist-test-slave-5k54 krb5kdc[28259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544500, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:40.067524 28389 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:21:40.067960 28389 webserver.cc:460] Webserver started at http://127.8.85.1:37031/ using document root <none> and password file <none>
I20260812 14:21:40.068189 28389 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:40.068255 28389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:40.068393 28389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:40.069442 28389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "89df0d733d60422098da5744ea123bd5"
format_stamp: "Formatted at 2026-08-12 14:21:40 on dist-test-slave-5k54"
server_key: "d427018f150a151d34a39308524428e5"
server_key_iv: "de5fa0842b94d8fa3e99c53244be481a"
server_key_version: "encryptionkey@0"
I20260812 14:21:40.069832 28389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "89df0d733d60422098da5744ea123bd5"
format_stamp: "Formatted at 2026-08-12 14:21:40 on dist-test-slave-5k54"
server_key: "d427018f150a151d34a39308524428e5"
server_key_iv: "de5fa0842b94d8fa3e99c53244be481a"
server_key_version: "encryptionkey@0"
I20260812 14:21:40.071167 28389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:21:40.072088 28409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:40.072288 28389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:21:40.072403 28389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "89df0d733d60422098da5744ea123bd5"
format_stamp: "Formatted at 2026-08-12 14:21:40 on dist-test-slave-5k54"
server_key: "d427018f150a151d34a39308524428e5"
server_key_iv: "de5fa0842b94d8fa3e99c53244be481a"
server_key_version: "encryptionkey@0"
I20260812 14:21:40.072489 28389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:40.085395 28389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:40.087127 28389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:40.087299 28389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:40.087738 28389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:40.088055 28389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:40.088090 28389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:40.088142 28389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:40.088168 28389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:40.094007 28389 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:33597
I20260812 14:21:40.094071 28542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:33597 every 8 connection(s)
I20260812 14:21:40.094388 28389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:21:40.096390 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 28389
I20260812 14:21:40.096516 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:21:40.096715 8532 external_mini_cluster.cc:1428] Setting key fe0d2ba53f203f371e89b922786e02cf
Aug 12 14:21:40 dist-test-slave-5k54 krb5kdc[28259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544500, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:21:40.104974 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.095118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:57601 (local address 127.8.85.62:36807)
0812 14:21:40.095161 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:40.095163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:40.095403 (+ 240us) server_negotiation.cc:407] Connection header received
0812 14:21:40.095971 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:40.095972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:40.095985 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:40.096050 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:40.097216 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.097931 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.098901 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.099240 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.099319 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.101909 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:40.101911 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:40.101911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:40.101917 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:40.103399 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:40.103951 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:40.103952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:40.103952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:40.104006 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:40.104306 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:40.104306 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:40.104306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:40.104408 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:40.104458 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:40.104818 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 14:21:40.104906 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:21:40.105238 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.095129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:36807 (local address 127.8.85.1:57601)
0812 14:21:40.095266 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:40.095302 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:40.095839 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:40.096120 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:40.096125 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:40.096848 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:40.097135 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.097142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.098026 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.098027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.098803 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.098804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.099347 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.099348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.099533 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.100038 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:21:40.100049 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:40.101799 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:40.103517 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:40.103526 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:40.103528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:40.103872 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:40.104075 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:40.104076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:40.104076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:40.104235 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:40.104476 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:40.104482 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:40.104665 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 14:21:40.104804 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 14:21:40.104988 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":68,"threads_started":1}
I20260812 14:21:40.105865 28543 heartbeater.cc:347] Connected to a master server at 127.8.85.62:36807
I20260812 14:21:40.105973 28543 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:40.106184 28543 heartbeater.cc:511] Master 127.8.85.62:36807 requested a full tablet report, sending...
I20260812 14:21:40.106807 28316 ts_manager.cc:194] Registered new tserver with Master: 89df0d733d60422098da5744ea123bd5 (127.8.85.1:33597)
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
I20260812 14:21:40.107692 28316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:57601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:21:40.126274 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:36807
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:40.208586 28548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:40.208786 28548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:40.208809 28548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:40.209888 28548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:40.209924 28548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:40.209950 28548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:21:40.211334 28548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:36807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:40.211582 28548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:40.211771 28548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:40.216682 28555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:40.216697 28554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:40.216697 28559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:40.216956 28548 server_base.cc:1034] running on GCE node
I20260812 14:21:40.217206 28548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:40.217507 28548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:40.218674 28548 hybrid_clock.cc:648] HybridClock initialized: now 1786544500218658 us; error 47 us; skew 500 ppm
Aug 12 14:21:40 dist-test-slave-5k54 krb5kdc[28259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544500, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:40.221531 28548 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:21:40.221974 28548 webserver.cc:460] Webserver started at http://127.8.85.2:43307/ using document root <none> and password file <none>
I20260812 14:21:40.222205 28548 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:40.222250 28548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:40.222414 28548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:40.223507 28548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "b42379c01bed4dd38adb5b1d6c5dcec1"
format_stamp: "Formatted at 2026-08-12 14:21:40 on dist-test-slave-5k54"
server_key: "d31d07faf81ef549937da7f4f819171e"
server_key_iv: "e431432b50d49411bee45e560c5c1d14"
server_key_version: "encryptionkey@0"
I20260812 14:21:40.223871 28548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "b42379c01bed4dd38adb5b1d6c5dcec1"
format_stamp: "Formatted at 2026-08-12 14:21:40 on dist-test-slave-5k54"
server_key: "d31d07faf81ef549937da7f4f819171e"
server_key_iv: "e431432b50d49411bee45e560c5c1d14"
server_key_version: "encryptionkey@0"
I20260812 14:21:40.225191 28548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:21:40.225858 28566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:40.226035 28548 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:21:40.226138 28548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "b42379c01bed4dd38adb5b1d6c5dcec1"
format_stamp: "Formatted at 2026-08-12 14:21:40 on dist-test-slave-5k54"
server_key: "d31d07faf81ef549937da7f4f819171e"
server_key_iv: "e431432b50d49411bee45e560c5c1d14"
server_key_version: "encryptionkey@0"
I20260812 14:21:40.226215 28548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:40.235168 28548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:40.237164 28548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:40.237322 28548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:40.237541 28548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:40.237839 28548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:40.237901 28548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:40.237942 28548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:40.237970 28548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:40.243613 28548 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:33423
I20260812 14:21:40.243672 28694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:33423 every 8 connection(s)
I20260812 14:21:40.243964 28548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:21:40 dist-test-slave-5k54 krb5kdc[28259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544500, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:21:40.251748 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 28548
I20260812 14:21:40.251858 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:21:40.252059 8532 external_mini_cluster.cc:1428] Setting key f9372dd0d234df63b9578dded2333d34
I20260812 14:21:40.253093 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.244649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:43319 (local address 127.8.85.62:36807)
0812 14:21:40.244712 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:40.244714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:40.244916 (+ 202us) server_negotiation.cc:407] Connection header received
0812 14:21:40.245458 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:40.245460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:40.245474 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:40.245527 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:40.246446 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.246961 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.248012 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.248217 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.248297 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.250311 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:40.250313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:40.250313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:40.250322 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:40.251632 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:40.252109 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:40.252110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:40.252111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:40.252176 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:40.252452 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:40.252453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:40.252453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:40.252570 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:40.252647 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:40.252917 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 14:21:40.253002 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:21:40.253319 28697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.244617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:36807 (local address 127.8.85.2:43319)
0812 14:21:40.244789 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:40.244830 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:40.245315 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:40.245598 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:40.245601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:40.246153 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:40.246334 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.246339 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.247059 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.247060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.247921 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.247924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.248302 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.248303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.248430 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.248840 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:21:40.248849 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:40.250199 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:40.251722 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:40.251729 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:40.251731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:40.252032 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:40.252255 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:40.252256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:40.252256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:40.252374 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:40.252637 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:40.252644 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:40.252783 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 14:21:40.252894 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 14:21:40.253052 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260812 14:21:40.253958 28695 heartbeater.cc:347] Connected to a master server at 127.8.85.62:36807
I20260812 14:21:40.254065 28695 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:40.254236 28695 heartbeater.cc:511] Master 127.8.85.62:36807 requested a full tablet report, sending...
I20260812 14:21:40.254662 28316 ts_manager.cc:194] Registered new tserver with Master: b42379c01bed4dd38adb5b1d6c5dcec1 (127.8.85.2:33423)
I20260812 14:21:40.255069 28316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:43319
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:21:40.279202 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:36807
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:40.361833 28702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:40.362066 28702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:40.362107 28702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:40.363236 28702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:40.363304 28702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:40.363348 28702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:21:40.364813 28702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:36807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:40.365068 28702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:40.365294 28702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:40.370301 28711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:40.370321 28714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:40.370416 28702 server_base.cc:1034] running on GCE node
W20260812 14:21:40.370546 28709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:40.370843 28702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:40.371117 28702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:40.372268 28702 hybrid_clock.cc:648] HybridClock initialized: now 1786544500372245 us; error 35 us; skew 500 ppm
Aug 12 14:21:40 dist-test-slave-5k54 krb5kdc[28259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544500, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:40.374858 28702 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:21:40.375291 28702 webserver.cc:460] Webserver started at http://127.8.85.3:40487/ using document root <none> and password file <none>
I20260812 14:21:40.375582 28702 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:40.375653 28702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:40.375784 28702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:40.376767 28702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "11ddb14f6ada414eb0f225fd4ccbb873"
format_stamp: "Formatted at 2026-08-12 14:21:40 on dist-test-slave-5k54"
server_key: "ccd09963e3f3bf86f37ef485a754a2fa"
server_key_iv: "4985cc5c5f8bae0b5391564dd83825fe"
server_key_version: "encryptionkey@0"
I20260812 14:21:40.377074 28702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "11ddb14f6ada414eb0f225fd4ccbb873"
format_stamp: "Formatted at 2026-08-12 14:21:40 on dist-test-slave-5k54"
server_key: "ccd09963e3f3bf86f37ef485a754a2fa"
server_key_iv: "4985cc5c5f8bae0b5391564dd83825fe"
server_key_version: "encryptionkey@0"
I20260812 14:21:40.378322 28702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:21:40.379002 28721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:40.379182 28702 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:21:40.379273 28702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "11ddb14f6ada414eb0f225fd4ccbb873"
format_stamp: "Formatted at 2026-08-12 14:21:40 on dist-test-slave-5k54"
server_key: "ccd09963e3f3bf86f37ef485a754a2fa"
server_key_iv: "4985cc5c5f8bae0b5391564dd83825fe"
server_key_version: "encryptionkey@0"
I20260812 14:21:40.379329 28702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:40.390810 28702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:40.392728 28702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:40.392838 28702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:40.393016 28702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:40.393256 28702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:40.393285 28702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:40.393306 28702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:40.393318 28702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:40.399230 28702 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:32891
I20260812 14:21:40.399307 28859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:32891 every 8 connection(s)
I20260812 14:21:40.399631 28702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:21:40.404414 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 28702
I20260812 14:21:40.404546 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:21:40.404780 8532 external_mini_cluster.cc:1428] Setting key e6fab349c9d995acd954deaf8d7e88d0
Aug 12 14:21:40 dist-test-slave-5k54 krb5kdc[28259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544500, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:21:40.408890 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.400371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:38785 (local address 127.8.85.62:36807)
0812 14:21:40.400432 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:40.400434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:40.400659 (+ 225us) server_negotiation.cc:407] Connection header received
0812 14:21:40.401124 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:40.401125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:40.401134 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:40.401193 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:40.402074 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.402564 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.403572 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.403822 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.403881 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.406340 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:40.406342 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:40.406342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:40.406348 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:40.407788 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:40.408152 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:40.408153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:40.408153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:40.408205 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:40.408409 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:40.408410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:40.408410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:40.408506 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:40.408554 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:40.408757 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 14:21:40.408813 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:21:40.409070 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.400344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:36807 (local address 127.8.85.3:38785)
0812 14:21:40.400534 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:40.400575 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:40.401045 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:40.401245 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:40.401249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:40.401803 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:40.402000 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.402005 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.402645 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.402646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.403483 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.403486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.403901 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.403902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.404008 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.404454 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:21:40.404465 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:40.406254 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:40.407858 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:40.407864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:40.407866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:40.408085 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:40.408263 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:40.408263 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:40.408263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:40.408355 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:40.408559 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:40.408563 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:40.408668 (+ 105us) client_negotiation.cc:769] Sending connection context
0812 14:21:40.408745 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 14:21:40.408858 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":111,"threads_started":1}
I20260812 14:21:40.409543 28860 heartbeater.cc:347] Connected to a master server at 127.8.85.62:36807
I20260812 14:21:40.409641 28860 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:40.409819 28860 heartbeater.cc:511] Master 127.8.85.62:36807 requested a full tablet report, sending...
I20260812 14:21:40.410200 28316 ts_manager.cc:194] Registered new tserver with Master: 11ddb14f6ada414eb0f225fd4ccbb873 (127.8.85.3:32891)
I20260812 14:21:40.410575 28316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:38785
I20260812 14:21:40.416494 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:21:40.423645 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.417817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40060 (local address 127.8.85.62:36807)
0812 14:21:40.417898 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:40.417900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:40.418010 (+ 110us) server_negotiation.cc:407] Connection header received
0812 14:21:40.418071 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:40.418072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:40.418080 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:40.418127 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:40.418715 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.419191 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.420069 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.420352 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.420413 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.421544 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:40.421545 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:40.421545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:40.421550 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:40.422700 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:40.423006 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:40.423007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:40.423007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:40.423056 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:40.423184 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:40.423184 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:40.423185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:40.423305 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:40.423380 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:40.423527 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 14:21:40.423589 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 14:21:40.425155 28316 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:21:40.426736 28316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:21:40.433307 28872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.428912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:32891 (local address 127.0.0.1:60486)
0812 14:21:40.429086 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:40.429125 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:40.429197 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:40.429803 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:40.429803 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:40.429806 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:40.429854 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.429856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.430910 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.430912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.432204 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.432207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.433022 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.433023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.433081 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.433092 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 14:21:40.433178 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 14:21:40.433233 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":52,"threads_started":1}
I20260812 14:21:40.433320 28877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.429370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.8.85.3:32891)
0812 14:21:40.429550 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:40.429554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:40.429565 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:21:40.429604 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:40.429606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:40.429693 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:40.429932 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:40.430018 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.430794 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.432280 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.432939 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.433019 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.433076 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:40.433197 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 14:21:40.433242 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":115,"threads_started":1}
I20260812 14:21:40.433665 28873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.428914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:33597 (local address 127.0.0.1:38930)
0812 14:21:40.429562 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:40.429573 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:40.429620 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:40.429873 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:40.429874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:40.429876 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:40.429919 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.429920 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.431147 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.431148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.432123 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.432125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.433437 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.433438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.433502 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.433506 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:21:40.433559 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:21:40.433602 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":2,"thread_start_us":49,"threads_started":1}
I20260812 14:21:40.433928 28878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.429417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38930 (local address 127.8.85.1:33597)
0812 14:21:40.429620 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:40.429625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:40.429635 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:21:40.429690 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:40.429692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:40.429709 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:40.429791 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:40.429999 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.431044 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.432210 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.432942 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.433019 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.433076 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:40.433776 (+ 700us) server_negotiation.cc:299] Negotiation successful
0812 14:21:40.433827 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":82,"threads_started":1}
I20260812 14:21:40.434122 28871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.428913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:33423 (local address 127.0.0.1:46572)
0812 14:21:40.429295 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:40.429305 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:40.429355 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:40.429646 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:40.429649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:40.429652 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:40.429723 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.429725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.431106 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.431107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.432220 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.432221 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.433898 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.433899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.433964 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.433968 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:21:40.434023 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 14:21:40.434060 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":24,"thread_start_us":77,"threads_started":1}
I20260812 14:21:40.434188 28782 tablet_service.cc:1467] Processing CreateTablet for tablet 241d8f2bae0942ba9501bf2c7480c3df (DEFAULT_TABLE table=test-table [id=3e21e8a27215428aaf4b6e6568a66611]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:40.434369 28875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.429092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46572 (local address 127.8.85.2:33423)
0812 14:21:40.429254 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:40.429258 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:40.429368 (+ 110us) server_negotiation.cc:407] Connection header received
0812 14:21:40.429440 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:40.429443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:40.429460 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:40.429546 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:40.429812 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.430999 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.433114 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.433808 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.433884 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.433938 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:40.434239 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 14:21:40.434291 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
I20260812 14:21:40.434552 28782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241d8f2bae0942ba9501bf2c7480c3df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:40.434567 28466 tablet_service.cc:1467] Processing CreateTablet for tablet 241d8f2bae0942ba9501bf2c7480c3df (DEFAULT_TABLE table=test-table [id=3e21e8a27215428aaf4b6e6568a66611]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:40.434891 28466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241d8f2bae0942ba9501bf2c7480c3df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:40.434931 28621 tablet_service.cc:1467] Processing CreateTablet for tablet 241d8f2bae0942ba9501bf2c7480c3df (DEFAULT_TABLE table=test-table [id=3e21e8a27215428aaf4b6e6568a66611]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:40.435166 28621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241d8f2bae0942ba9501bf2c7480c3df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:40.437798 28879 tablet_bootstrap.cc:492] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1: Bootstrap starting.
I20260812 14:21:40.438251 28880 tablet_bootstrap.cc:492] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873: Bootstrap starting.
I20260812 14:21:40.438514 28879 tablet_bootstrap.cc:654] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:40.438895 28880 tablet_bootstrap.cc:654] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:40.439072 28879 log.cc:824] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:40.439397 28880 log.cc:824] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:40.439625 28881 tablet_bootstrap.cc:492] T 241d8f2bae0942ba9501bf2c7480c3df P 89df0d733d60422098da5744ea123bd5: Bootstrap starting.
I20260812 14:21:40.439985 28879 tablet_bootstrap.cc:492] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1: No bootstrap required, opened a new log
I20260812 14:21:40.440066 28879 ts_tablet_manager.cc:1397] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:21:40.440200 28880 tablet_bootstrap.cc:492] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873: No bootstrap required, opened a new log
I20260812 14:21:40.440263 28880 ts_tablet_manager.cc:1397] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:21:40.440326 28881 tablet_bootstrap.cc:654] T 241d8f2bae0942ba9501bf2c7480c3df P 89df0d733d60422098da5744ea123bd5: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:40.440776 28881 log.cc:824] T 241d8f2bae0942ba9501bf2c7480c3df P 89df0d733d60422098da5744ea123bd5: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:40.441483 28881 tablet_bootstrap.cc:492] T 241d8f2bae0942ba9501bf2c7480c3df P 89df0d733d60422098da5744ea123bd5: No bootstrap required, opened a new log
I20260812 14:21:40.441568 28881 ts_tablet_manager.cc:1397] T 241d8f2bae0942ba9501bf2c7480c3df P 89df0d733d60422098da5744ea123bd5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:21:40.442247 28879 raft_consensus.cc:348] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11ddb14f6ada414eb0f225fd4ccbb873" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 32891 } } peers { permanent_uuid: "b42379c01bed4dd38adb5b1d6c5dcec1" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33423 } } peers { permanent_uuid: "89df0d733d60422098da5744ea123bd5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33597 } }
I20260812 14:21:40.442202 28880 raft_consensus.cc:348] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11ddb14f6ada414eb0f225fd4ccbb873" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 32891 } } peers { permanent_uuid: "b42379c01bed4dd38adb5b1d6c5dcec1" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33423 } } peers { permanent_uuid: "89df0d733d60422098da5744ea123bd5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33597 } }
I20260812 14:21:40.442397 28880 raft_consensus.cc:374] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:40.442397 28879 raft_consensus.cc:374] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:40.442436 28880 raft_consensus.cc:729] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11ddb14f6ada414eb0f225fd4ccbb873, State: Initialized, Role: FOLLOWER
I20260812 14:21:40.442437 28879 raft_consensus.cc:729] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b42379c01bed4dd38adb5b1d6c5dcec1, State: Initialized, Role: FOLLOWER
I20260812 14:21:40.442584 28880 consensus_queue.cc:260] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11ddb14f6ada414eb0f225fd4ccbb873" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 32891 } } peers { permanent_uuid: "b42379c01bed4dd38adb5b1d6c5dcec1" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33423 } } peers { permanent_uuid: "89df0d733d60422098da5744ea123bd5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33597 } }
I20260812 14:21:40.442584 28879 consensus_queue.cc:260] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11ddb14f6ada414eb0f225fd4ccbb873" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 32891 } } peers { permanent_uuid: "b42379c01bed4dd38adb5b1d6c5dcec1" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33423 } } peers { permanent_uuid: "89df0d733d60422098da5744ea123bd5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33597 } }
I20260812 14:21:40.442883 28879 ts_tablet_manager.cc:1428] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:21:40.442873 28880 ts_tablet_manager.cc:1428] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:21:40.443053 28860 heartbeater.cc:503] Master 127.8.85.62:36807 was elected leader, sending a full tablet report...
I20260812 14:21:40.443066 28695 heartbeater.cc:503] Master 127.8.85.62:36807 was elected leader, sending a full tablet report...
I20260812 14:21:40.443271 28881 raft_consensus.cc:348] T 241d8f2bae0942ba9501bf2c7480c3df P 89df0d733d60422098da5744ea123bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11ddb14f6ada414eb0f225fd4ccbb873" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 32891 } } peers { permanent_uuid: "b42379c01bed4dd38adb5b1d6c5dcec1" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33423 } } peers { permanent_uuid: "89df0d733d60422098da5744ea123bd5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33597 } }
I20260812 14:21:40.443470 28881 raft_consensus.cc:374] T 241d8f2bae0942ba9501bf2c7480c3df P 89df0d733d60422098da5744ea123bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:40.443518 28881 raft_consensus.cc:729] T 241d8f2bae0942ba9501bf2c7480c3df P 89df0d733d60422098da5744ea123bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89df0d733d60422098da5744ea123bd5, State: Initialized, Role: FOLLOWER
I20260812 14:21:40.443641 28881 consensus_queue.cc:260] T 241d8f2bae0942ba9501bf2c7480c3df P 89df0d733d60422098da5744ea123bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11ddb14f6ada414eb0f225fd4ccbb873" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 32891 } } peers { permanent_uuid: "b42379c01bed4dd38adb5b1d6c5dcec1" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33423 } } peers { permanent_uuid: "89df0d733d60422098da5744ea123bd5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33597 } }
I20260812 14:21:40.443938 28881 ts_tablet_manager.cc:1428] T 241d8f2bae0942ba9501bf2c7480c3df P 89df0d733d60422098da5744ea123bd5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:21:40.443953 28543 heartbeater.cc:503] Master 127.8.85.62:36807 was elected leader, sending a full tablet report...
W20260812 14:21:40.494544 28696 tablet.cc:2369] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:21:40.569151 28886 raft_consensus.cc:482] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:21:40.569273 28886 raft_consensus.cc:504] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11ddb14f6ada414eb0f225fd4ccbb873" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 32891 } } peers { permanent_uuid: "b42379c01bed4dd38adb5b1d6c5dcec1" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33423 } } peers { permanent_uuid: "89df0d733d60422098da5744ea123bd5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33597 } }
I20260812 14:21:40.569734 28886 leader_election.cc:288] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11ddb14f6ada414eb0f225fd4ccbb873 (127.8.85.3:32891), 89df0d733d60422098da5744ea123bd5 (127.8.85.1:33597)
I20260812 14:21:40.572553 28877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.570040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:41887 (local address 127.8.85.3:32891)
0812 14:21:40.570092 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:40.570094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:40.570104 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:21:40.570137 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:40.570138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:40.570153 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:40.570233 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:40.570427 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.570899 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.571835 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.572206 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.572290 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.572301 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:40.572448 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 14:21:40.572487 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:21:40.572552 28697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.569894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:32891 (local address 127.8.85.2:41887)
0812 14:21:40.569953 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:40.569964 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:40.570007 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:40.570242 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:40.570244 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:40.570246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:40.570323 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.570325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.570985 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.570986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.571728 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.571730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.572288 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.572289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.572370 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.572373 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:21:40.572442 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 14:21:40.572483 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 14:21:40.572717 28878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.570038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:39695 (local address 127.8.85.1:33597)
0812 14:21:40.570082 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:40.570083 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:40.570195 (+ 112us) server_negotiation.cc:407] Connection header received
0812 14:21:40.570263 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:40.570264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:40.570277 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:40.570322 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:40.570486 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.570952 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.571965 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.572371 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.572463 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.572472 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:40.572617 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 14:21:40.572655 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:21:40.572727 28889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.569949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:33597 (local address 127.8.85.2:39695)
0812 14:21:40.570119 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:40.570128 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:40.570209 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:40.570333 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:40.570333 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:40.570335 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:40.570418 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.570419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.571041 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.571042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.571858 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:40.571861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.572464 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.572465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:40.572542 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.572545 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:21:40.572612 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 14:21:40.572649 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":121,"threads_started":1}
I20260812 14:21:40.572929 28806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241d8f2bae0942ba9501bf2c7480c3df" candidate_uuid: "b42379c01bed4dd38adb5b1d6c5dcec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11ddb14f6ada414eb0f225fd4ccbb873" is_pre_election: true
I20260812 14:21:40.573122 28806 raft_consensus.cc:2454] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b42379c01bed4dd38adb5b1d6c5dcec1 in term 0.
I20260812 14:21:40.573148 28491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241d8f2bae0942ba9501bf2c7480c3df" candidate_uuid: "b42379c01bed4dd38adb5b1d6c5dcec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89df0d733d60422098da5744ea123bd5" is_pre_election: true
I20260812 14:21:40.573302 28491 raft_consensus.cc:2454] T 241d8f2bae0942ba9501bf2c7480c3df P 89df0d733d60422098da5744ea123bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b42379c01bed4dd38adb5b1d6c5dcec1 in term 0.
I20260812 14:21:40.573367 28569 leader_election.cc:302] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11ddb14f6ada414eb0f225fd4ccbb873, b42379c01bed4dd38adb5b1d6c5dcec1; no voters:
I20260812 14:21:40.573488 28886 raft_consensus.cc:2781] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:21:40.573567 28886 raft_consensus.cc:482] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:21:40.573602 28886 raft_consensus.cc:3037] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:40.574324 28886 raft_consensus.cc:504] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11ddb14f6ada414eb0f225fd4ccbb873" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 32891 } } peers { permanent_uuid: "b42379c01bed4dd38adb5b1d6c5dcec1" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33423 } } peers { permanent_uuid: "89df0d733d60422098da5744ea123bd5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33597 } }
I20260812 14:21:40.574476 28886 leader_election.cc:288] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [CANDIDATE]: Term 1 election: Requested vote from peers 11ddb14f6ada414eb0f225fd4ccbb873 (127.8.85.3:32891), 89df0d733d60422098da5744ea123bd5 (127.8.85.1:33597)
I20260812 14:21:40.574609 28491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241d8f2bae0942ba9501bf2c7480c3df" candidate_uuid: "b42379c01bed4dd38adb5b1d6c5dcec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89df0d733d60422098da5744ea123bd5"
I20260812 14:21:40.574656 28806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241d8f2bae0942ba9501bf2c7480c3df" candidate_uuid: "b42379c01bed4dd38adb5b1d6c5dcec1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11ddb14f6ada414eb0f225fd4ccbb873"
I20260812 14:21:40.574717 28491 raft_consensus.cc:3037] T 241d8f2bae0942ba9501bf2c7480c3df P 89df0d733d60422098da5744ea123bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:40.574754 28806 raft_consensus.cc:3037] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:40.575582 28806 raft_consensus.cc:2454] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b42379c01bed4dd38adb5b1d6c5dcec1 in term 1.
I20260812 14:21:40.575677 28491 raft_consensus.cc:2454] T 241d8f2bae0942ba9501bf2c7480c3df P 89df0d733d60422098da5744ea123bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b42379c01bed4dd38adb5b1d6c5dcec1 in term 1.
I20260812 14:21:40.575789 28569 leader_election.cc:302] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11ddb14f6ada414eb0f225fd4ccbb873, b42379c01bed4dd38adb5b1d6c5dcec1; no voters:
I20260812 14:21:40.575891 28886 raft_consensus.cc:2781] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:40.576056 28886 raft_consensus.cc:686] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [term 1 LEADER]: Becoming Leader. State: Replica: b42379c01bed4dd38adb5b1d6c5dcec1, State: Running, Role: LEADER
I20260812 14:21:40.576155 28886 consensus_queue.cc:237] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11ddb14f6ada414eb0f225fd4ccbb873" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 32891 } } peers { permanent_uuid: "b42379c01bed4dd38adb5b1d6c5dcec1" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33423 } } peers { permanent_uuid: "89df0d733d60422098da5744ea123bd5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33597 } }
I20260812 14:21:40.577047 28315 catalog_manager.cc:5495] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 reported cstate change: term changed from 0 to 1, leader changed from <none> to b42379c01bed4dd38adb5b1d6c5dcec1 (127.8.85.2). New cstate: current_term: 1 leader_uuid: "b42379c01bed4dd38adb5b1d6c5dcec1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11ddb14f6ada414eb0f225fd4ccbb873" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 32891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b42379c01bed4dd38adb5b1d6c5dcec1" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89df0d733d60422098da5744ea123bd5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33597 } health_report { overall_health: UNKNOWN } } }
W20260812 14:21:40.595117 28544 tablet.cc:2369] T 241d8f2bae0942ba9501bf2c7480c3df P 89df0d733d60422098da5744ea123bd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:21:40.617313 28875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.614923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46584 (local address 127.8.85.2:33423)
0812 14:21:40.614990 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:40.614992 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:40.615001 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:21:40.615067 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:40.615068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:40.615088 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:40.615149 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:21:40.615311 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.615838 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.616571 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.616753 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.616842 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.617029 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:21:40.617128 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:21:40.617188 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:40.617217 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 14:21:40.617236 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:21:40.619488 28491 raft_consensus.cc:1261] T 241d8f2bae0942ba9501bf2c7480c3df P 89df0d733d60422098da5744ea123bd5 [term 1 FOLLOWER]: Refusing update from remote peer b42379c01bed4dd38adb5b1d6c5dcec1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:21:40.619488 28806 raft_consensus.cc:1261] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873 [term 1 FOLLOWER]: Refusing update from remote peer b42379c01bed4dd38adb5b1d6c5dcec1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:21:40.619877 28886 consensus_queue.cc:1035] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11ddb14f6ada414eb0f225fd4ccbb873" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 32891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:21:40.620044 28890 consensus_queue.cc:1035] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89df0d733d60422098da5744ea123bd5" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 33597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:21:40.622571 28895 mvcc.cc:205] Tried to move back new op lower bound from 7317686274534236160 to 7317686274360487936. Current Snapshot: MvccSnapshot[applied={T|T < 7317686274534236160}]
I20260812 14:21:40.623154 28894 mvcc.cc:205] Tried to move back new op lower bound from 7317686274534236160 to 7317686274360487936. Current Snapshot: MvccSnapshot[applied={T|T < 7317686274534236160}]
I20260812 14:21:40.623486 28897 mvcc.cc:205] Tried to move back new op lower bound from 7317686274534236160 to 7317686274360487936. Current Snapshot: MvccSnapshot[applied={T|T < 7317686274534236160}]
I20260812 14:21:40.625182 28315 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:21:40.625284 28315 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:21:40.626070 28315 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b73bf1f9df5b44aabf4635cc7142ef45: Sending DeleteTablet for 3 replicas of tablet 241d8f2bae0942ba9501bf2c7480c3df
I20260812 14:21:40.626425 28466 tablet_service.cc:1514] Processing DeleteTablet for tablet 241d8f2bae0942ba9501bf2c7480c3df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:40 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:38930
I20260812 14:21:40.626466 28782 tablet_service.cc:1514] Processing DeleteTablet for tablet 241d8f2bae0942ba9501bf2c7480c3df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:40 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:60486
I20260812 14:21:40.626729 28621 tablet_service.cc:1514] Processing DeleteTablet for tablet 241d8f2bae0942ba9501bf2c7480c3df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:40 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:46572
I20260812 14:21:40.626762 28909 tablet_replica.cc:330] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873: stopping tablet replica
I20260812 14:21:40.626858 28909 raft_consensus.cc:2229] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:21:40.626943 28910 tablet_replica.cc:330] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1: stopping tablet replica
I20260812 14:21:40.626951 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 28389
I20260812 14:21:40.626979 28909 raft_consensus.cc:2258] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:40.634078 28909 ts_tablet_manager.cc:1905] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:21:40.635852 28909 ts_tablet_manager.cc:1918] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:21:40.635932 28909 log.cc:1196] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/241d8f2bae0942ba9501bf2c7480c3df
I20260812 14:21:40.636214 28909 ts_tablet_manager.cc:1939] T 241d8f2bae0942ba9501bf2c7480c3df P 11ddb14f6ada414eb0f225fd4ccbb873: Deleting consensus metadata
I20260812 14:21:40.636649 28298 catalog_manager.cc:4841] TS 11ddb14f6ada414eb0f225fd4ccbb873 (127.8.85.3:32891): tablet 241d8f2bae0942ba9501bf2c7480c3df (table test-table [id=3e21e8a27215428aaf4b6e6568a66611]) successfully deleted
I20260812 14:21:40.643532 28910 raft_consensus.cc:2229] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:21:40.643702 28910 raft_consensus.cc:2258] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:40.644161 28910 ts_tablet_manager.cc:1905] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 14:21:40.645594 28297 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:33597 (error 108)
I20260812 14:21:40.645656 28910 ts_tablet_manager.cc:1918] T 241d8f2bae0942ba9501bf2c7480c3df P b42379c01bed4dd38adb5b1d6c5dcec1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:21:40.645849 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 28548
I20260812 14:21:40.649423 28871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.649298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:33597 (local address 127.0.0.1:38938)
0812 14:21:40.649341 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:40.649385 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:33597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 14:21:40.649864 28297 catalog_manager.cc:4584] TS 89df0d733d60422098da5744ea123bd5 (127.8.85.1:33597): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 241d8f2bae0942ba9501bf2c7480c3df: Network error: Client connection negotiation failed: client connection to 127.8.85.1:33597: connect: Connection refused (error 111)
I20260812 14:21:40.653082 28871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.652957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:33423 (local address 127.0.0.1:46596)
0812 14:21:40.653017 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:40.653045 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.2:33423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 14:21:40.653082 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 28702
I20260812 14:21:40.661180 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 28277
2026-08-12T14:21:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2133 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T14:21:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:21:40Z Disabled control of system clock
I20260812 14:21:40.692704 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:33887
--webserver_interface=127.8.85.62
--webserver_port=0
--builtin_ntp_servers=127.8.85.20:41323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:33887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:21:40.775134 28916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:40.775349 28916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:40.775372 28916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:40.776520 28916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:40.776563 28916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:40.776576 28916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:40.776589 28916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:40.776602 28916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:40.777858 28916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:33887
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:33887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:40.778047 28916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:40.778276 28916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:40.782944 28922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:40.782964 28923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:40.782984 28925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:40.783147 28916 server_base.cc:1034] running on GCE node
I20260812 14:21:40.783474 28916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:40.783777 28916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:40.784938 28916 hybrid_clock.cc:648] HybridClock initialized: now 1786544500784915 us; error 44 us; skew 500 ppm
I20260812 14:21:40.786209 28916 webserver.cc:460] Webserver started at http://127.8.85.62:39101/ using document root <none> and password file <none>
I20260812 14:21:40.786465 28916 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:40.786535 28916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:40.786679 28916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:40.787763 28916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "52a0483b1f8a4a33bb13d215e8bc3970"
format_stamp: "Formatted at 2026-08-12 14:21:40 on dist-test-slave-5k54"
server_key: "219c820f7ef72e2eca296d3b13666f34"
server_key_iv: "cbc132887782a4a8c18182c29a5424cd"
server_key_version: "encryptionkey@0"
I20260812 14:21:40.788185 28916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "52a0483b1f8a4a33bb13d215e8bc3970"
format_stamp: "Formatted at 2026-08-12 14:21:40 on dist-test-slave-5k54"
server_key: "219c820f7ef72e2eca296d3b13666f34"
server_key_iv: "cbc132887782a4a8c18182c29a5424cd"
server_key_version: "encryptionkey@0"
I20260812 14:21:40.789449 28916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:21:40.790167 28933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:40.790349 28916 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:21:40.790449 28916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "52a0483b1f8a4a33bb13d215e8bc3970"
format_stamp: "Formatted at 2026-08-12 14:21:40 on dist-test-slave-5k54"
server_key: "219c820f7ef72e2eca296d3b13666f34"
server_key_iv: "cbc132887782a4a8c18182c29a5424cd"
server_key_version: "encryptionkey@0"
I20260812 14:21:40.790520 28916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:40.810482 28916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:40.810824 28916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:40.810994 28916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:40.815238 28916 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:33887
I20260812 14:21:40.815274 28993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:33887 every 8 connection(s)
I20260812 14:21:40.815688 28916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:40.816733 28994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:40.816944 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 28916
I20260812 14:21:40.817054 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:21:40.817279 8532 external_mini_cluster.cc:1428] Setting key 0bb6a82554dd0404e0034711394c451e
I20260812 14:21:40.820355 28994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970: Bootstrap starting.
I20260812 14:21:40.821103 28994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:40.821399 28994 log.cc:824] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:40.822126 28994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970: No bootstrap required, opened a new log
I20260812 14:21:40.823258 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:40.817854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35600 (local address 127.8.85.62:33887)
0812 14:21:40.818060 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:40.818068 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:40.818093 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:21:40.818700 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:40.818708 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:40.818808 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:40.818934 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:40.819160 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.820094 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.821202 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:40.821688 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:40.821837 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:40.822604 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:40.822609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:40.822609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:40.822781 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:40.822814 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:40.822824 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:40.822934 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:40.823012 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 14:21:40.823096 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260812 14:21:40.824360 28994 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52a0483b1f8a4a33bb13d215e8bc3970" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33887 } }
I20260812 14:21:40.824510 28994 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:40.824532 28994 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52a0483b1f8a4a33bb13d215e8bc3970, State: Initialized, Role: FOLLOWER
I20260812 14:21:40.824679 28994 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52a0483b1f8a4a33bb13d215e8bc3970" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33887 } }
I20260812 14:21:40.824745 28994 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:40.824806 28994 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:40.824855 28994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:40.825475 28994 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52a0483b1f8a4a33bb13d215e8bc3970" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33887 } }
I20260812 14:21:40.825592 28994 leader_election.cc:302] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52a0483b1f8a4a33bb13d215e8bc3970; no voters:
I20260812 14:21:40.825780 28994 leader_election.cc:288] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:21:40.825879 29001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:40.826069 29001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [term 1 LEADER]: Becoming Leader. State: Replica: 52a0483b1f8a4a33bb13d215e8bc3970, State: Running, Role: LEADER
I20260812 14:21:40.826085 28994 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:40.826236 29001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52a0483b1f8a4a33bb13d215e8bc3970" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33887 } }
I20260812 14:21:40.826614 29003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52a0483b1f8a4a33bb13d215e8bc3970. Latest consensus state: current_term: 1 leader_uuid: "52a0483b1f8a4a33bb13d215e8bc3970" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52a0483b1f8a4a33bb13d215e8bc3970" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33887 } } }
I20260812 14:21:40.826609 29002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52a0483b1f8a4a33bb13d215e8bc3970" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52a0483b1f8a4a33bb13d215e8bc3970" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 33887 } } }
I20260812 14:21:40.826710 29003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:40.826717 29002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:40.826984 29011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:40.827529 29011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:40.829274 29011 catalog_manager.cc:1347] Generated new cluster ID: fa4f9ec78f034eeaa0cfbcc30dd91519
I20260812 14:21:40.829326 29011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:21:40.853369 29011 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:21:40.854090 29011 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:40.864380 29011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970: Generated new TSK 0
I20260812 14:21:40.864589 29011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:21:40.871809 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33887
--builtin_ntp_servers=127.8.85.20:41323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:21:40.957332 29022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:40.957541 29022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:40.957563 29022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:40.958756 29022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:40.958796 29022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:40.958822 29022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:21:40.960250 29022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:40.960512 29022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:40.960813 29022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:40.965996 29028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:40.966001 29031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:40.966115 29029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:40.966404 29022 server_base.cc:1034] running on GCE node
I20260812 14:21:40.966619 29022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:40.966842 29022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:40.968014 29022 hybrid_clock.cc:648] HybridClock initialized: now 1786544500968000 us; error 24 us; skew 500 ppm
I20260812 14:21:40.969261 29022 webserver.cc:460] Webserver started at http://127.8.85.1:41249/ using document root <none> and password file <none>
I20260812 14:21:40.969525 29022 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:40.969596 29022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:40.969748 29022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:40.970798 29022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "e25eb3f5dddd4a59819c454e1c7d31a2"
format_stamp: "Formatted at 2026-08-12 14:21:40 on dist-test-slave-5k54"
server_key: "cc7760d0e979f3472658b789cfd3721b"
server_key_iv: "595d6b83486163f8f086017843b469d8"
server_key_version: "encryptionkey@0"
I20260812 14:21:40.971161 29022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "e25eb3f5dddd4a59819c454e1c7d31a2"
format_stamp: "Formatted at 2026-08-12 14:21:40 on dist-test-slave-5k54"
server_key: "cc7760d0e979f3472658b789cfd3721b"
server_key_iv: "595d6b83486163f8f086017843b469d8"
server_key_version: "encryptionkey@0"
I20260812 14:21:40.972545 29022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:21:40.973275 29037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:40.973450 29022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:21:40.973551 29022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "e25eb3f5dddd4a59819c454e1c7d31a2"
format_stamp: "Formatted at 2026-08-12 14:21:40 on dist-test-slave-5k54"
server_key: "cc7760d0e979f3472658b789cfd3721b"
server_key_iv: "595d6b83486163f8f086017843b469d8"
server_key_version: "encryptionkey@0"
I20260812 14:21:40.973634 29022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:40.997983 29022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:40.998291 29022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:40.998440 29022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:40.998697 29022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:40.999050 29022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:40.999130 29022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:40.999177 29022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:40.999205 29022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:41.005177 29022 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:35009
I20260812 14:21:41.005231 29180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:35009 every 8 connection(s)
I20260812 14:21:41.005564 29022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:21:41.009094 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 29022
I20260812 14:21:41.009218 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:21:41.009435 8532 external_mini_cluster.cc:1428] Setting key e65d4afac353d96d0c729da3e5f95831
I20260812 14:21:41.010433 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33887
--builtin_ntp_servers=127.8.85.20:41323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 14:21:41.011322 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.006318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:45063 (local address 127.8.85.62:33887)
0812 14:21:41.006389 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:41.006392 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:41.006640 (+ 248us) server_negotiation.cc:407] Connection header received
0812 14:21:41.007189 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:41.007191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:41.007209 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:41.007275 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:41.008034 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.008556 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.009746 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.010001 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.010079 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.011045 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:41.011046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:41.011047 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:41.011051 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:41.011066 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:41.011070 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:41.011113 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:41.011201 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 14:21:41.011244 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:21:41.011561 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.006288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33887 (local address 127.8.85.1:45063)
0812 14:21:41.006512 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:41.006548 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:41.007076 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:41.007356 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:41.007359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:41.007367 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:41.007897 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.007901 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.008659 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.008661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.009647 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.009651 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.010105 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.010106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.010258 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.010880 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:21:41.010881 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:41.010911 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:21:41.010934 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:41.011124 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:41.011129 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:41.011132 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:21:41.011201 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 14:21:41.011301 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":103,"threads_started":1}
I20260812 14:21:41.012072 29181 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33887
I20260812 14:21:41.012190 29181 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:41.012380 29181 heartbeater.cc:511] Master 127.8.85.62:33887 requested a full tablet report, sending...
I20260812 14:21:41.013000 28952 ts_manager.cc:194] Registered new tserver with Master: e25eb3f5dddd4a59819c454e1c7d31a2 (127.8.85.1:35009)
I20260812 14:21:41.013764 28952 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.85.1:45063
W20260812 14:21:41.094417 29185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:41.094616 29185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:41.094638 29185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:41.095728 29185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:41.095762 29185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:41.095786 29185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:21:41.097049 29185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:41.097229 29185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:41.097465 29185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:41.102523 29194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:41.102548 29190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:41.102707 29185 server_base.cc:1034] running on GCE node
W20260812 14:21:41.102509 29191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:41.103021 29185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:41.103276 29185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:41.104444 29185 hybrid_clock.cc:648] HybridClock initialized: now 1786544501104433 us; error 33 us; skew 500 ppm
I20260812 14:21:41.105787 29185 webserver.cc:460] Webserver started at http://127.8.85.2:46199/ using document root <none> and password file <none>
I20260812 14:21:41.106048 29185 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:41.106123 29185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:41.106276 29185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:41.107378 29185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "4b40036e64894d029fa1aedf72ea9222"
format_stamp: "Formatted at 2026-08-12 14:21:41 on dist-test-slave-5k54"
server_key: "00650e09625df6e33ed5d005640c0def"
server_key_iv: "c4b23050f9d91dc9983dcff2e310268e"
server_key_version: "encryptionkey@0"
I20260812 14:21:41.107851 29185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "4b40036e64894d029fa1aedf72ea9222"
format_stamp: "Formatted at 2026-08-12 14:21:41 on dist-test-slave-5k54"
server_key: "00650e09625df6e33ed5d005640c0def"
server_key_iv: "c4b23050f9d91dc9983dcff2e310268e"
server_key_version: "encryptionkey@0"
I20260812 14:21:41.109248 29185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:21:41.109964 29202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:41.110136 29185 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:21:41.110237 29185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "4b40036e64894d029fa1aedf72ea9222"
format_stamp: "Formatted at 2026-08-12 14:21:41 on dist-test-slave-5k54"
server_key: "00650e09625df6e33ed5d005640c0def"
server_key_iv: "c4b23050f9d91dc9983dcff2e310268e"
server_key_version: "encryptionkey@0"
I20260812 14:21:41.110318 29185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:41.117980 29185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:41.118265 29185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:41.118422 29185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:41.118633 29185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:41.118950 29185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:41.119006 29185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:41.119047 29185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:41.119076 29185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:41.125062 29185 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:33563
I20260812 14:21:41.125110 29340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:33563 every 8 connection(s)
I20260812 14:21:41.125470 29185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:21:41.126972 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 29185
I20260812 14:21:41.127096 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:21:41.127331 8532 external_mini_cluster.cc:1428] Setting key 2a4f24234877dcc914fffa2f4e2627c5
I20260812 14:21:41.128373 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33887
--builtin_ntp_servers=127.8.85.20:41323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 14:21:41.131832 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.126274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:37727 (local address 127.8.85.62:33887)
0812 14:21:41.126404 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:41.126407 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:41.126643 (+ 236us) server_negotiation.cc:407] Connection header received
0812 14:21:41.127462 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:41.127463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:41.127475 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:41.127544 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:41.128450 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.129176 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.130388 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.130576 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.130642 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.131590 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:41.131591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:41.131591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:41.131595 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:41.131610 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:41.131614 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:41.131652 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:41.131754 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 14:21:41.131775 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 14:21:41.132114 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.126264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33887 (local address 127.8.85.2:37727)
0812 14:21:41.126520 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:41.126554 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:41.127308 (+ 754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:41.127697 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:41.127700 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:41.127709 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:41.128351 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.128356 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.129270 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.129271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.130279 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.130282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.130672 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.130673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.130836 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.131395 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:21:41.131395 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:41.131467 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:21:41.131497 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:41.131673 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:41.131679 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:41.131682 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:21:41.131752 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 14:21:41.131859 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":96,"threads_started":1}
I20260812 14:21:41.132647 29341 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33887
I20260812 14:21:41.132741 29341 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:41.132942 29341 heartbeater.cc:511] Master 127.8.85.62:33887 requested a full tablet report, sending...
I20260812 14:21:41.133333 28952 ts_manager.cc:194] Registered new tserver with Master: 4b40036e64894d029fa1aedf72ea9222 (127.8.85.2:33563)
I20260812 14:21:41.133711 28952 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.85.2:37727
W20260812 14:21:41.214010 29344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:41.214303 29344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:41.214347 29344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:41.215590 29344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:41.215659 29344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:41.215700 29344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:21:41.217090 29344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:33887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:41.217365 29344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:41.217672 29344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:41.222916 29350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:41.222931 29353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:41.222983 29351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:41.223227 29344 server_base.cc:1034] running on GCE node
I20260812 14:21:41.223515 29344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:41.223762 29344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:41.224917 29344 hybrid_clock.cc:648] HybridClock initialized: now 1786544501224897 us; error 36 us; skew 500 ppm
I20260812 14:21:41.226454 29344 webserver.cc:460] Webserver started at http://127.8.85.3:37351/ using document root <none> and password file <none>
I20260812 14:21:41.226764 29344 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:41.226816 29344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:41.226950 29344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:41.228171 29344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe"
format_stamp: "Formatted at 2026-08-12 14:21:41 on dist-test-slave-5k54"
server_key: "eb0fb3cac835a8f9f828057ea616a80d"
server_key_iv: "0cafaefbef9f4dac98faad20b40f77aa"
server_key_version: "encryptionkey@0"
I20260812 14:21:41.228549 29344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe"
format_stamp: "Formatted at 2026-08-12 14:21:41 on dist-test-slave-5k54"
server_key: "eb0fb3cac835a8f9f828057ea616a80d"
server_key_iv: "0cafaefbef9f4dac98faad20b40f77aa"
server_key_version: "encryptionkey@0"
I20260812 14:21:41.229828 29344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:21:41.230561 29360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:41.230721 29344 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:21:41.230822 29344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe"
format_stamp: "Formatted at 2026-08-12 14:21:41 on dist-test-slave-5k54"
server_key: "eb0fb3cac835a8f9f828057ea616a80d"
server_key_iv: "0cafaefbef9f4dac98faad20b40f77aa"
server_key_version: "encryptionkey@0"
I20260812 14:21:41.230906 29344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:41.245782 29344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:41.246070 29344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:41.246227 29344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:41.246457 29344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:41.246779 29344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:41.246840 29344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:41.246881 29344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:41.246909 29344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:41.252856 29344 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:44821
I20260812 14:21:41.252904 29494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:44821 every 8 connection(s)
I20260812 14:21:41.253217 29344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:21:41.254822 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 29344
I20260812 14:21:41.254949 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:21:41.255151 8532 external_mini_cluster.cc:1428] Setting key c12599e0e21f82d3d2022f548c3c8227
I20260812 14:21:41.258772 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.253929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:45911 (local address 127.8.85.62:33887)
0812 14:21:41.253998 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:41.254001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:41.254301 (+ 300us) server_negotiation.cc:407] Connection header received
0812 14:21:41.254838 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:41.254839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:41.254852 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:41.254910 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:41.255784 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.256355 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.257360 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.257558 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.257637 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.258495 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:41.258496 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:41.258497 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:41.258501 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:41.258515 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:41.258520 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:41.258586 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:41.258669 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 14:21:41.258694 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:21:41.258996 29497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.253973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:33887 (local address 127.8.85.3:45911)
0812 14:21:41.254164 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:41.254198 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:41.254731 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:41.254983 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:41.254988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:41.254998 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:41.255687 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.255694 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.256461 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.256462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.257267 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.257269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.257643 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.257645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.257796 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.258340 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:21:41.258340 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:41.258368 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:21:41.258391 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:41.258589 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:41.258595 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:41.258598 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:21:41.258670 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:21:41.258772 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260812 14:21:41.259512 29495 heartbeater.cc:347] Connected to a master server at 127.8.85.62:33887
I20260812 14:21:41.259593 29495 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:41.259786 29495 heartbeater.cc:511] Master 127.8.85.62:33887 requested a full tablet report, sending...
I20260812 14:21:41.260155 28952 ts_manager.cc:194] Registered new tserver with Master: 3ce05d06aeee414b87ce2c3ccf49cdbe (127.8.85.3:44821)
I20260812 14:21:41.260524 28952 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.85.3:45911
I20260812 14:21:41.267010 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:21:41.271098 28999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.268442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35614 (local address 127.8.85.62:33887)
0812 14:21:41.268496 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:41.268497 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:41.268626 (+ 129us) server_negotiation.cc:407] Connection header received
0812 14:21:41.268680 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:41.268681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:41.268689 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:41.268739 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:41.268906 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.269353 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.270108 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.270283 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.270334 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.270905 (+ 571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:41.270906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:41.270906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:41.270910 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:41.270922 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:41.270926 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:41.270967 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:41.271028 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 14:21:41.271047 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:21:41.272611 28952 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:21:41.274109 28952 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:21:41.280659 29508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.276625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:35009 (local address 127.0.0.1:51694)
0812 14:21:41.276851 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:41.276883 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:41.276948 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:41.277257 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:41.277257 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:41.277259 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:41.277333 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.277335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.278621 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.278622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.279515 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.279522 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.280370 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.280371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.280448 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.280462 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 14:21:41.280538 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 14:21:41.280589 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":2,"thread_start_us":83,"threads_started":1}
I20260812 14:21:41.280673 29509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.276842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51694 (local address 127.8.85.1:35009)
0812 14:21:41.277057 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:41.277061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:41.277070 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:21:41.277135 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:41.277137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:41.277153 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:41.277237 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:41.277435 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.278538 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.279612 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.280286 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.280378 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.280470 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:41.280552 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 14:21:41.280589 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":100,"threads_started":1}
I20260812 14:21:41.281270 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.276710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:44821 (local address 127.0.0.1:40930)
0812 14:21:41.277325 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:41.277335 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:41.277404 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:41.277584 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:41.277585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:41.277586 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:41.277626 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.277627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.278658 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.278659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.279979 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.279982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.280999 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.280999 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.281064 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.281071 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 14:21:41.281124 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:21:41.281171 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":124,"thread_start_us":90,"threads_started":1}
I20260812 14:21:41.281281 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.276410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:33563 (local address 127.0.0.1:59396)
0812 14:21:41.276827 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:41.276867 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:41.276950 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:41.277157 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:41.277159 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:41.277162 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:41.277229 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.277231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.278612 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.278613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.279979 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.279982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.280996 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.280997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.281064 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.281070 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 14:21:41.281126 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 14:21:41.281171 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":73,"thread_start_us":74,"threads_started":1}
I20260812 14:21:41.281656 29100 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1cf1e84b5b402c938212a6e9833e9c (DEFAULT_TABLE table=test-table [id=1895614439e8436f84e899300e3f5797]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:41.282012 29100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1cf1e84b5b402c938212a6e9833e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:41.283566 29505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.276500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59396 (local address 127.8.85.2:33563)
0812 14:21:41.276760 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:41.276765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:41.276934 (+ 169us) server_negotiation.cc:407] Connection header received
0812 14:21:41.277041 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:41.277043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:41.277059 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:41.277129 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:41.277306 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.278506 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.280090 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.280799 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.280849 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.280936 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:41.281145 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 14:21:41.283480 (+ 2335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":106,"threads_started":1}
I20260812 14:21:41.283833 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.276559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40930 (local address 127.8.85.3:44821)
0812 14:21:41.276758 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:41.276763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:41.277403 (+ 640us) server_negotiation.cc:407] Connection header received
0812 14:21:41.277480 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:41.277483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:41.277497 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:41.277585 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:41.277687 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.278589 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.280090 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.280799 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.280849 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.280926 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:41.283712 (+ 2786us) server_negotiation.cc:299] Negotiation successful
0812 14:21:41.283758 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":285,"threads_started":1}
I20260812 14:21:41.284257 29265 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1cf1e84b5b402c938212a6e9833e9c (DEFAULT_TABLE table=test-table [id=1895614439e8436f84e899300e3f5797]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:41.284417 29416 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1cf1e84b5b402c938212a6e9833e9c (DEFAULT_TABLE table=test-table [id=1895614439e8436f84e899300e3f5797]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:41.284610 29265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1cf1e84b5b402c938212a6e9833e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:41.284675 29416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1cf1e84b5b402c938212a6e9833e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:41.285137 29511 tablet_bootstrap.cc:492] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2: Bootstrap starting.
I20260812 14:21:41.285835 29511 tablet_bootstrap.cc:654] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:41.286176 29511 log.cc:824] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:41.287001 29511 tablet_bootstrap.cc:492] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2: No bootstrap required, opened a new log
I20260812 14:21:41.287062 29511 ts_tablet_manager.cc:1397] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:21:41.287341 29513 tablet_bootstrap.cc:492] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222: Bootstrap starting.
I20260812 14:21:41.288061 29513 tablet_bootstrap.cc:654] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:41.288373 29513 log.cc:824] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:41.289077 29513 tablet_bootstrap.cc:492] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222: No bootstrap required, opened a new log
I20260812 14:21:41.289156 29513 ts_tablet_manager.cc:1397] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:21:41.289605 29511 raft_consensus.cc:348] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b40036e64894d029fa1aedf72ea9222" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33563 } } peers { permanent_uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44821 } } peers { permanent_uuid: "e25eb3f5dddd4a59819c454e1c7d31a2" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35009 } }
I20260812 14:21:41.289754 29511 raft_consensus.cc:374] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:41.289777 29511 raft_consensus.cc:729] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e25eb3f5dddd4a59819c454e1c7d31a2, State: Initialized, Role: FOLLOWER
I20260812 14:21:41.289901 29511 consensus_queue.cc:260] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b40036e64894d029fa1aedf72ea9222" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33563 } } peers { permanent_uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44821 } } peers { permanent_uuid: "e25eb3f5dddd4a59819c454e1c7d31a2" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35009 } }
I20260812 14:21:41.290187 29511 ts_tablet_manager.cc:1428] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:21:41.290236 29181 heartbeater.cc:503] Master 127.8.85.62:33887 was elected leader, sending a full tablet report...
I20260812 14:21:41.290390 29514 tablet_bootstrap.cc:492] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe: Bootstrap starting.
I20260812 14:21:41.290977 29514 tablet_bootstrap.cc:654] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:41.291316 29514 log.cc:824] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:41.291646 29513 raft_consensus.cc:348] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b40036e64894d029fa1aedf72ea9222" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33563 } } peers { permanent_uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44821 } } peers { permanent_uuid: "e25eb3f5dddd4a59819c454e1c7d31a2" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35009 } }
I20260812 14:21:41.291823 29513 raft_consensus.cc:374] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:41.291862 29513 raft_consensus.cc:729] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b40036e64894d029fa1aedf72ea9222, State: Initialized, Role: FOLLOWER
I20260812 14:21:41.291976 29513 consensus_queue.cc:260] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b40036e64894d029fa1aedf72ea9222" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33563 } } peers { permanent_uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44821 } } peers { permanent_uuid: "e25eb3f5dddd4a59819c454e1c7d31a2" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35009 } }
I20260812 14:21:41.292143 29514 tablet_bootstrap.cc:492] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe: No bootstrap required, opened a new log
I20260812 14:21:41.292217 29514 ts_tablet_manager.cc:1397] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:21:41.292237 29513 ts_tablet_manager.cc:1428] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:21:41.292280 29341 heartbeater.cc:503] Master 127.8.85.62:33887 was elected leader, sending a full tablet report...
I20260812 14:21:41.294297 29514 raft_consensus.cc:348] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b40036e64894d029fa1aedf72ea9222" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33563 } } peers { permanent_uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44821 } } peers { permanent_uuid: "e25eb3f5dddd4a59819c454e1c7d31a2" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35009 } }
I20260812 14:21:41.294485 29514 raft_consensus.cc:374] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:41.294512 29514 raft_consensus.cc:729] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce05d06aeee414b87ce2c3ccf49cdbe, State: Initialized, Role: FOLLOWER
I20260812 14:21:41.294611 29514 consensus_queue.cc:260] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b40036e64894d029fa1aedf72ea9222" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33563 } } peers { permanent_uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44821 } } peers { permanent_uuid: "e25eb3f5dddd4a59819c454e1c7d31a2" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35009 } }
I20260812 14:21:41.294826 29514 ts_tablet_manager.cc:1428] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:21:41.294916 29495 heartbeater.cc:503] Master 127.8.85.62:33887 was elected leader, sending a full tablet report...
W20260812 14:21:41.376050 29342 tablet.cc:2369] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:21:41.378377 29518 raft_consensus.cc:482] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:21:41.378482 29518 raft_consensus.cc:504] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b40036e64894d029fa1aedf72ea9222" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33563 } } peers { permanent_uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44821 } } peers { permanent_uuid: "e25eb3f5dddd4a59819c454e1c7d31a2" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35009 } }
I20260812 14:21:41.378790 29518 leader_election.cc:288] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b40036e64894d029fa1aedf72ea9222 (127.8.85.2:33563), 3ce05d06aeee414b87ce2c3ccf49cdbe (127.8.85.3:44821)
I20260812 14:21:41.381860 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.379083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:34153 (local address 127.8.85.3:44821)
0812 14:21:41.379161 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:41.379163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:41.379175 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:21:41.379209 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:41.379210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:41.379221 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:41.379279 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:41.379520 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.380132 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.381112 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.381493 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.381546 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.381555 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:41.381739 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 14:21:41.381786 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 14:21:41.381872 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.378983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:44821 (local address 127.8.85.1:34153)
0812 14:21:41.379059 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:41.379073 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:41.379134 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:41.379299 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:41.379299 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:41.379301 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:41.379395 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.379397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.380227 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.380228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.381015 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.381017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.381582 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.381583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.381657 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.381661 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:21:41.381715 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:21:41.381749 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":4}
I20260812 14:21:41.381938 29505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.379082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:52349 (local address 127.8.85.2:33563)
0812 14:21:41.379127 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:41.379129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:41.379269 (+ 140us) server_negotiation.cc:407] Connection header received
0812 14:21:41.379301 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:41.379302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:41.379312 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:41.379372 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:41.379588 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.380049 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.381172 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.381515 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.381607 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.381615 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:41.381812 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 14:21:41.381858 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:21:41.381949 29522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.378983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:33563 (local address 127.8.85.1:52349)
0812 14:21:41.379164 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:41.379174 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:41.379240 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:41.379397 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:41.379397 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:41.379399 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:41.379513 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.379515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.380120 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.380121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.381044 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:41.381046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.381626 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.381631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:41.381721 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.381726 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:21:41.381786 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:21:41.381820 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":58,"thread_start_us":99,"threads_started":1}
I20260812 14:21:41.382220 29438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1cf1e84b5b402c938212a6e9833e9c" candidate_uuid: "e25eb3f5dddd4a59819c454e1c7d31a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe" is_pre_election: true
I20260812 14:21:41.382300 29287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1cf1e84b5b402c938212a6e9833e9c" candidate_uuid: "e25eb3f5dddd4a59819c454e1c7d31a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b40036e64894d029fa1aedf72ea9222" is_pre_election: true
I20260812 14:21:41.382382 29438 raft_consensus.cc:2454] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e25eb3f5dddd4a59819c454e1c7d31a2 in term 0.
I20260812 14:21:41.382467 29287 raft_consensus.cc:2454] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e25eb3f5dddd4a59819c454e1c7d31a2 in term 0.
I20260812 14:21:41.382651 29041 leader_election.cc:302] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce05d06aeee414b87ce2c3ccf49cdbe, e25eb3f5dddd4a59819c454e1c7d31a2; no voters:
I20260812 14:21:41.382778 29518 raft_consensus.cc:2781] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:21:41.382838 29518 raft_consensus.cc:482] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:21:41.382861 29518 raft_consensus.cc:3037] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:41.383641 29518 raft_consensus.cc:504] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b40036e64894d029fa1aedf72ea9222" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33563 } } peers { permanent_uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44821 } } peers { permanent_uuid: "e25eb3f5dddd4a59819c454e1c7d31a2" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35009 } }
I20260812 14:21:41.383785 29518 leader_election.cc:288] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [CANDIDATE]: Term 1 election: Requested vote from peers 4b40036e64894d029fa1aedf72ea9222 (127.8.85.2:33563), 3ce05d06aeee414b87ce2c3ccf49cdbe (127.8.85.3:44821)
I20260812 14:21:41.383926 29438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1cf1e84b5b402c938212a6e9833e9c" candidate_uuid: "e25eb3f5dddd4a59819c454e1c7d31a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe"
I20260812 14:21:41.383942 29287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1cf1e84b5b402c938212a6e9833e9c" candidate_uuid: "e25eb3f5dddd4a59819c454e1c7d31a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b40036e64894d029fa1aedf72ea9222"
I20260812 14:21:41.384013 29438 raft_consensus.cc:3037] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:41.384013 29287 raft_consensus.cc:3037] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:41.384966 29287 raft_consensus.cc:2454] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e25eb3f5dddd4a59819c454e1c7d31a2 in term 1.
I20260812 14:21:41.384967 29438 raft_consensus.cc:2454] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e25eb3f5dddd4a59819c454e1c7d31a2 in term 1.
I20260812 14:21:41.385138 29041 leader_election.cc:302] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ce05d06aeee414b87ce2c3ccf49cdbe, e25eb3f5dddd4a59819c454e1c7d31a2; no voters:
I20260812 14:21:41.385242 29518 raft_consensus.cc:2781] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:41.385401 29518 raft_consensus.cc:686] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [term 1 LEADER]: Becoming Leader. State: Replica: e25eb3f5dddd4a59819c454e1c7d31a2, State: Running, Role: LEADER
I20260812 14:21:41.385511 29518 consensus_queue.cc:237] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b40036e64894d029fa1aedf72ea9222" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33563 } } peers { permanent_uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44821 } } peers { permanent_uuid: "e25eb3f5dddd4a59819c454e1c7d31a2" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35009 } }
I20260812 14:21:41.386404 28952 catalog_manager.cc:5495] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e25eb3f5dddd4a59819c454e1c7d31a2 (127.8.85.1). New cstate: current_term: 1 leader_uuid: "e25eb3f5dddd4a59819c454e1c7d31a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b40036e64894d029fa1aedf72ea9222" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e25eb3f5dddd4a59819c454e1c7d31a2" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35009 } health_report { overall_health: HEALTHY } } }
I20260812 14:21:41.397485 29509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.395341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51710 (local address 127.8.85.1:35009)
0812 14:21:41.395393 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:41.395394 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:41.395443 (+ 49us) server_negotiation.cc:407] Connection header received
0812 14:21:41.395559 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:41.395560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:41.395575 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:41.395634 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:21:41.395745 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.396174 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.396866 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:41.397045 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:41.397096 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:41.397219 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:21:41.397307 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:21:41.397365 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:41.397404 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 14:21:41.397420 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:21:41.399106 29287 raft_consensus.cc:1261] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222 [term 1 FOLLOWER]: Refusing update from remote peer e25eb3f5dddd4a59819c454e1c7d31a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:21:41.399137 29438 raft_consensus.cc:1261] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe [term 1 FOLLOWER]: Refusing update from remote peer e25eb3f5dddd4a59819c454e1c7d31a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:21:41.399477 29523 consensus_queue.cc:1035] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b40036e64894d029fa1aedf72ea9222" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 33563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:21:41.399608 29518 consensus_queue.cc:1035] T 0a1cf1e84b5b402c938212a6e9833e9c P e25eb3f5dddd4a59819c454e1c7d31a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ce05d06aeee414b87ce2c3ccf49cdbe" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:21:41.402107 29530 mvcc.cc:205] Tried to move back new op lower bound from 7317686277728370688 to 7317686277675831296. Current Snapshot: MvccSnapshot[applied={T|T < 7317686277728370688}]
I20260812 14:21:41.402209 29529 mvcc.cc:205] Tried to move back new op lower bound from 7317686277728370688 to 7317686277675831296. Current Snapshot: MvccSnapshot[applied={T|T < 7317686277728370688}]
I20260812 14:21:41.404527 28951 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:21:41.404606 28951 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:21:41.405405 28951 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52a0483b1f8a4a33bb13d215e8bc3970: Sending DeleteTablet for 3 replicas of tablet 0a1cf1e84b5b402c938212a6e9833e9c
I20260812 14:21:41.405745 29265 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1cf1e84b5b402c938212a6e9833e9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:41 UTC) from {username='slave'} at 127.0.0.1:59396
I20260812 14:21:41.405778 29416 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1cf1e84b5b402c938212a6e9833e9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:41 UTC) from {username='slave'} at 127.0.0.1:40930
I20260812 14:21:41.405862 29100 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1cf1e84b5b402c938212a6e9833e9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:41 UTC) from {username='slave'} at 127.0.0.1:51694
I20260812 14:21:41.406037 29538 tablet_replica.cc:330] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222: stopping tablet replica
I20260812 14:21:41.406036 29539 tablet_replica.cc:330] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe: stopping tablet replica
I20260812 14:21:41.406136 29539 raft_consensus.cc:2229] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:21:41.406162 29538 raft_consensus.cc:2229] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:21:41.406210 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 29022
I20260812 14:21:41.406256 29539 raft_consensus.cc:2258] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:41.406401 29538 raft_consensus.cc:2258] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:41.407087 29538 ts_tablet_manager.cc:1905] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:21:41.409169 29538 ts_tablet_manager.cc:1918] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:21:41.409248 29538 log.cc:1196] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/0a1cf1e84b5b402c938212a6e9833e9c
I20260812 14:21:41.409502 29538 ts_tablet_manager.cc:1939] T 0a1cf1e84b5b402c938212a6e9833e9c P 4b40036e64894d029fa1aedf72ea9222: Deleting consensus metadata
I20260812 14:21:41.409951 28937 catalog_manager.cc:4841] TS 4b40036e64894d029fa1aedf72ea9222 (127.8.85.2:33563): tablet 0a1cf1e84b5b402c938212a6e9833e9c (table test-table [id=1895614439e8436f84e899300e3f5797]) successfully deleted
I20260812 14:21:41.410774 29539 ts_tablet_manager.cc:1905] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:21:41.412420 29539 ts_tablet_manager.cc:1918] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:21:41.412498 29539 log.cc:1196] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/0a1cf1e84b5b402c938212a6e9833e9c
I20260812 14:21:41.412784 29539 ts_tablet_manager.cc:1939] T 0a1cf1e84b5b402c938212a6e9833e9c P 3ce05d06aeee414b87ce2c3ccf49cdbe: Deleting consensus metadata
I20260812 14:21:41.413244 28936 catalog_manager.cc:4841] TS 3ce05d06aeee414b87ce2c3ccf49cdbe (127.8.85.3:44821): tablet 0a1cf1e84b5b402c938212a6e9833e9c (table test-table [id=1895614439e8436f84e899300e3f5797]) successfully deleted
W20260812 14:21:41.418203 28937 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:35009 (error 108)
I20260812 14:21:41.418520 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 29185
I20260812 14:21:41.418722 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:41.418579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:35009 (local address 127.0.0.1:51712)
0812 14:21:41.418641 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:41.418685 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:35009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 14:21:41.418838 28937 catalog_manager.cc:4584] TS e25eb3f5dddd4a59819c454e1c7d31a2 (127.8.85.1:35009): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a1cf1e84b5b402c938212a6e9833e9c: Network error: Client connection negotiation failed: client connection to 127.8.85.1:35009: connect: Connection refused (error 111)
I20260812 14:21:41.424345 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 29344
I20260812 14:21:41.430823 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 28916
2026-08-12T14:21:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (768 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:21:41 dist-test-slave-5k54 krb5kdc[29546](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:21:41 dist-test-slave-5k54 krb5kdc[29546](info): set up 2 sockets
Aug 12 14:21:41 dist-test-slave-5k54 krb5kdc[29546](info): commencing operation
krb5kdc: starting...
W20260812 14:21:42.549006 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:21:42 dist-test-slave-5k54 krb5kdc[29546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:21:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:21:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:21:42.671106 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:43571
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:43571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:42.755371 29562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:42.755650 29562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:42.755674 29562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:42.756821 29562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:42.756863 29562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:42.756877 29562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:42.756891 29562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:42.756903 29562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:42.758374 29562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:43571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:43571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:42.758618 29562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:42.758867 29562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:42.763712 29568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:42.763698 29569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:42.763931 29562 server_base.cc:1034] running on GCE node
W20260812 14:21:42.763713 29572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:42.764235 29562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:42.764595 29562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:42.765780 29562 hybrid_clock.cc:648] HybridClock initialized: now 1786544502765767 us; error 30 us; skew 500 ppm
Aug 12 14:21:42 dist-test-slave-5k54 krb5kdc[29546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544502, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:42.768446 29562 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:21:42.768874 29562 webserver.cc:460] Webserver started at http://127.8.85.62:35935/ using document root <none> and password file <none>
I20260812 14:21:42.769097 29562 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:42.769165 29562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:42.769320 29562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:42.770388 29562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "c625b5cf9c9746ad839c332a9982a1a7"
format_stamp: "Formatted at 2026-08-12 14:21:42 on dist-test-slave-5k54"
server_key: "e65e776acade73e367fa5c2c0d21c366"
server_key_iv: "70b0a075b110b803dd369a6cf0aa0e5e"
server_key_version: "encryptionkey@0"
I20260812 14:21:42.770759 29562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "c625b5cf9c9746ad839c332a9982a1a7"
format_stamp: "Formatted at 2026-08-12 14:21:42 on dist-test-slave-5k54"
server_key: "e65e776acade73e367fa5c2c0d21c366"
server_key_iv: "70b0a075b110b803dd369a6cf0aa0e5e"
server_key_version: "encryptionkey@0"
I20260812 14:21:42.772176 29562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:21:42.772944 29580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:42.773118 29562 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:21:42.773231 29562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "c625b5cf9c9746ad839c332a9982a1a7"
format_stamp: "Formatted at 2026-08-12 14:21:42 on dist-test-slave-5k54"
server_key: "e65e776acade73e367fa5c2c0d21c366"
server_key_iv: "70b0a075b110b803dd369a6cf0aa0e5e"
server_key_version: "encryptionkey@0"
I20260812 14:21:42.773306 29562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:42.785377 29562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:42.787767 29562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:42.787945 29562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:42.791903 29562 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:43571
I20260812 14:21:42.791939 29639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:43571 every 8 connection(s)
I20260812 14:21:42.792289 29562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:42.793190 29640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:42.795554 29640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7: Bootstrap starting.
I20260812 14:21:42.796156 29640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:42.796455 29640 log.cc:824] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:42.796518 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 29562
I20260812 14:21:42.796655 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:21:42.796845 8532 external_mini_cluster.cc:1428] Setting key cc745d40e0f459c94dd07606270be94c
I20260812 14:21:42.797289 29640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7: No bootstrap required, opened a new log
I20260812 14:21:42.799084 29640 raft_consensus.cc:348] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c625b5cf9c9746ad839c332a9982a1a7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43571 } }
I20260812 14:21:42.799220 29640 raft_consensus.cc:374] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:42.799242 29640 raft_consensus.cc:729] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c625b5cf9c9746ad839c332a9982a1a7, State: Initialized, Role: FOLLOWER
I20260812 14:21:42.799367 29640 consensus_queue.cc:260] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c625b5cf9c9746ad839c332a9982a1a7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43571 } }
I20260812 14:21:42.799470 29640 raft_consensus.cc:388] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:42.799511 29640 raft_consensus.cc:482] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:42.799558 29640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:42.800396 29640 raft_consensus.cc:504] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c625b5cf9c9746ad839c332a9982a1a7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43571 } }
I20260812 14:21:42.800520 29640 leader_election.cc:302] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c625b5cf9c9746ad839c332a9982a1a7; no voters:
I20260812 14:21:42.800726 29640 leader_election.cc:288] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:21:42.800760 29645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:42.800865 29645 raft_consensus.cc:686] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [term 1 LEADER]: Becoming Leader. State: Replica: c625b5cf9c9746ad839c332a9982a1a7, State: Running, Role: LEADER
I20260812 14:21:42.800949 29645 consensus_queue.cc:237] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c625b5cf9c9746ad839c332a9982a1a7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43571 } }
I20260812 14:21:42.801005 29640 sys_catalog.cc:564] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:42.801307 29645 sys_catalog.cc:455] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c625b5cf9c9746ad839c332a9982a1a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c625b5cf9c9746ad839c332a9982a1a7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43571 } } }
I20260812 14:21:42.801379 29645 sys_catalog.cc:458] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:42.801512 29645 sys_catalog.cc:455] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c625b5cf9c9746ad839c332a9982a1a7. Latest consensus state: current_term: 1 leader_uuid: "c625b5cf9c9746ad839c332a9982a1a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c625b5cf9c9746ad839c332a9982a1a7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43571 } } }
I20260812 14:21:42.801584 29645 sys_catalog.cc:458] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:42.801869 29651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:42.802399 29651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:42.804194 29651 catalog_manager.cc:1347] Generated new cluster ID: a577190caa68409dbeeb0ef4ff1318e4
I20260812 14:21:42.804260 29651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:21:42 dist-test-slave-5k54 krb5kdc[29546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:21:42.810992 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:42.797414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40496 (local address 127.8.85.62:43571)
0812 14:21:42.797602 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:42.797610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:42.797635 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:21:42.799098 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:42.799108 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:42.799168 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:42.799407 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:42.800244 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:42.801641 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:42.803092 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:42.803800 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:42.803937 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:42.806961 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:42.806966 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:42.806967 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:42.806980 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:42.809169 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:42.809616 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:42.809619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:42.809620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:42.809756 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:42.810008 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:42.810009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:42.810009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:42.810327 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:42.810486 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:42.810608 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 14:21:42.810750 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
I20260812 14:21:42.828594 29651 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:21:42.829279 29651 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:42.841856 29651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7: Generated new TSK 0
I20260812 14:21:42.842108 29651 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:21:42.875861 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:43571
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:42.966415 29669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:42.966610 29669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:42.966634 29669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:42.967700 29669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:42.967734 29669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:42.967760 29669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:21:42.969133 29669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:43571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:42.969316 29669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:42.969478 29669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:42.974328 29676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:42.974406 29675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:42.974324 29678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:42.974757 29669 server_base.cc:1034] running on GCE node
I20260812 14:21:42.974957 29669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:42.975196 29669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:42.976352 29669 hybrid_clock.cc:648] HybridClock initialized: now 1786544502976341 us; error 65 us; skew 500 ppm
Aug 12 14:21:42 dist-test-slave-5k54 krb5kdc[29546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544502, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:42.978938 29669 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:21:42.979499 29669 webserver.cc:460] Webserver started at http://127.8.85.1:45957/ using document root <none> and password file <none>
I20260812 14:21:42.979755 29669 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:42.979807 29669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:42.979995 29669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:42.981083 29669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "40d60503e29d442fa1f4cfb66d0f2b9c"
format_stamp: "Formatted at 2026-08-12 14:21:42 on dist-test-slave-5k54"
server_key: "2911f2069f452bad26ff56e4015804f8"
server_key_iv: "6389059bf6a96dcac16e0616f457e499"
server_key_version: "encryptionkey@0"
I20260812 14:21:42.981487 29669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "40d60503e29d442fa1f4cfb66d0f2b9c"
format_stamp: "Formatted at 2026-08-12 14:21:42 on dist-test-slave-5k54"
server_key: "2911f2069f452bad26ff56e4015804f8"
server_key_iv: "6389059bf6a96dcac16e0616f457e499"
server_key_version: "encryptionkey@0"
I20260812 14:21:42.982909 29669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:21:42.983716 29687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:42.983909 29669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:21:42.984005 29669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "40d60503e29d442fa1f4cfb66d0f2b9c"
format_stamp: "Formatted at 2026-08-12 14:21:42 on dist-test-slave-5k54"
server_key: "2911f2069f452bad26ff56e4015804f8"
server_key_iv: "6389059bf6a96dcac16e0616f457e499"
server_key_version: "encryptionkey@0"
I20260812 14:21:42.984079 29669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:43.005435 29669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:43.007571 29669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:43.007736 29669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:43.007988 29669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:43.008322 29669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:43.008358 29669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:43.008381 29669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:43.008423 29669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:43.014462 29669 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:40511
I20260812 14:21:43.014535 29828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:40511 every 8 connection(s)
I20260812 14:21:43.014896 29669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:21:43 dist-test-slave-5k54 krb5kdc[29546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544502, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:21:43.022902 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 29669
I20260812 14:21:43.022990 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:21:43.023236 8532 external_mini_cluster.cc:1428] Setting key 033bd82cb56f01870cd57cce2b722ed2
I20260812 14:21:43.025230 29831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.015661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:43571 (local address 127.8.85.1:34429)
0812 14:21:43.015891 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:43.015942 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:43.016509 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:43.016782 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:43.016787 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:43.017358 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:43.017560 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.017566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.018327 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.018328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.019133 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.019135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.019651 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.019652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.019788 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.020266 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:21:43.020274 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:43.021694 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:43.023589 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:43.023598 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:43.023616 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:43.023944 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:43.024185 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:43.024186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:43.024187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:43.024343 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:43.024659 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:43.024665 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:43.024829 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 14:21:43.024923 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 14:21:43.025058 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":101,"threads_started":1}
I20260812 14:21:43.025233 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.015756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:34429 (local address 127.8.85.62:43571)
0812 14:21:43.015868 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:43.015870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:43.016042 (+ 172us) server_negotiation.cc:407] Connection header received
0812 14:21:43.016629 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:43.016630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:43.016643 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:43.016698 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:43.017668 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.018226 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.019237 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.019552 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.019636 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.021793 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:43.021795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:43.021795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:43.021805 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:43.023497 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:43.024041 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:43.024042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:43.024043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:43.024116 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:43.024436 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:43.024438 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:43.024438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:43.024588 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:43.024660 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:43.024970 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 14:21:43.025084 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 14:21:43.025810 29829 heartbeater.cc:347] Connected to a master server at 127.8.85.62:43571
I20260812 14:21:43.025944 29829 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:43.026166 29829 heartbeater.cc:511] Master 127.8.85.62:43571 requested a full tablet report, sending...
I20260812 14:21:43.026901 29599 ts_manager.cc:194] Registered new tserver with Master: 40d60503e29d442fa1f4cfb66d0f2b9c (127.8.85.1:40511)
I20260812 14:21:43.027841 29599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:34429
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:21:43.052767 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:43571
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:43.135548 29836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:43.135759 29836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:43.135797 29836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:43.136921 29836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:43.136986 29836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:43.137032 29836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:21:43.138463 29836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:43571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:43.138725 29836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:43.138958 29836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:43.143929 29847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:43.143905 29845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:43.143925 29844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:43.144227 29836 server_base.cc:1034] running on GCE node
I20260812 14:21:43.144450 29836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:43.144680 29836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:43.145841 29836 hybrid_clock.cc:648] HybridClock initialized: now 1786544503145829 us; error 30 us; skew 500 ppm
Aug 12 14:21:43 dist-test-slave-5k54 krb5kdc[29546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:43.148344 29836 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:21:43.148792 29836 webserver.cc:460] Webserver started at http://127.8.85.2:39851/ using document root <none> and password file <none>
I20260812 14:21:43.148988 29836 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:43.149029 29836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:43.149118 29836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:43.150089 29836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "92dbc197994943d4a36c4f6e50617ba7"
format_stamp: "Formatted at 2026-08-12 14:21:43 on dist-test-slave-5k54"
server_key: "a54c98928eacad5b70469ccc12735465"
server_key_iv: "ba38b9246acc048fd998740abb1ba8cb"
server_key_version: "encryptionkey@0"
I20260812 14:21:43.150388 29836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "92dbc197994943d4a36c4f6e50617ba7"
format_stamp: "Formatted at 2026-08-12 14:21:43 on dist-test-slave-5k54"
server_key: "a54c98928eacad5b70469ccc12735465"
server_key_iv: "ba38b9246acc048fd998740abb1ba8cb"
server_key_version: "encryptionkey@0"
I20260812 14:21:43.151700 29836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:21:43.152372 29854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:43.152556 29836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:43.152635 29836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "92dbc197994943d4a36c4f6e50617ba7"
format_stamp: "Formatted at 2026-08-12 14:21:43 on dist-test-slave-5k54"
server_key: "a54c98928eacad5b70469ccc12735465"
server_key_iv: "ba38b9246acc048fd998740abb1ba8cb"
server_key_version: "encryptionkey@0"
I20260812 14:21:43.152688 29836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:43.159211 29836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:43.161388 29836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:43.161497 29836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:43.161659 29836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:43.161891 29836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:43.161921 29836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:43.161940 29836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:43.161952 29836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:43.167533 29836 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:45867
I20260812 14:21:43.167588 29980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:45867 every 8 connection(s)
I20260812 14:21:43.167888 29836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:21:43.168157 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 29836
I20260812 14:21:43.168284 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:21:43.168473 8532 external_mini_cluster.cc:1428] Setting key 8f66b2b8a48687715a6cb6e638597e4f
Aug 12 14:21:43 dist-test-slave-5k54 krb5kdc[29546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
I20260812 14:21:43.178562 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.168587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:43177 (local address 127.8.85.62:43571)
0812 14:21:43.168660 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:43.168663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:43.169167 (+ 504us) server_negotiation.cc:407] Connection header received
0812 14:21:43.169682 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:43.169683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:43.169696 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:43.169764 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:43.170670 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.171333 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.172704 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.173048 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.173121 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.175542 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:43.175544 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:43.175545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:43.175554 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:43.177169 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:43.177567 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:43.177569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:43.177569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:43.177631 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:43.177880 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:43.177881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:43.177881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:43.177996 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:43.178071 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:43.178381 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 14:21:43.178473 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:21:43.178710 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.168581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:43571 (local address 127.8.85.2:43177)
0812 14:21:43.169037 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:43.169097 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:43.169591 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:43.169816 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:43.169818 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:43.170398 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:43.170579 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.170584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.171445 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.171447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.172610 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.172613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.173144 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.173145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.173243 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.173753 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:21:43.173761 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:43.175405 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:43.177263 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:43.177269 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:43.177272 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:43.177488 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:43.177705 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:43.177706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:43.177706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:43.177806 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:43.178062 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:43.178069 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:43.178196 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 14:21:43.178336 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 14:21:43.178492 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":86,"threads_started":1}
I20260812 14:21:43.179319 29981 heartbeater.cc:347] Connected to a master server at 127.8.85.62:43571
I20260812 14:21:43.179498 29981 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:43.179656 29981 heartbeater.cc:511] Master 127.8.85.62:43571 requested a full tablet report, sending...
I20260812 14:21:43.180073 29599 ts_manager.cc:194] Registered new tserver with Master: 92dbc197994943d4a36c4f6e50617ba7 (127.8.85.2:45867)
I20260812 14:21:43.180446 29599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:43177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:21:43.197656 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:43571
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:43.281296 29987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:43.281486 29987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:43.281508 29987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:43.282555 29987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:43.282588 29987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:43.282614 29987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:21:43.284065 29987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:43571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:43.284246 29987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:43.284427 29987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:43.289300 29995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:43.289315 29992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:43.289309 29993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:43.289523 29987 server_base.cc:1034] running on GCE node
I20260812 14:21:43.289739 29987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:43.289983 29987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:43.291124 29987 hybrid_clock.cc:648] HybridClock initialized: now 1786544503291103 us; error 35 us; skew 500 ppm
Aug 12 14:21:43 dist-test-slave-5k54 krb5kdc[29546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:43.293613 29987 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:21:43.294080 29987 webserver.cc:460] Webserver started at http://127.8.85.3:36797/ using document root <none> and password file <none>
I20260812 14:21:43.294348 29987 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:43.294422 29987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:43.294576 29987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:43.295678 29987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "846b09b7734043ff872c79d6710c35f1"
format_stamp: "Formatted at 2026-08-12 14:21:43 on dist-test-slave-5k54"
server_key: "1145d83d0e984fa172c8219676b0f3a7"
server_key_iv: "fa1b271bfb75304a3cc588d4f0e9bcc3"
server_key_version: "encryptionkey@0"
I20260812 14:21:43.296033 29987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "846b09b7734043ff872c79d6710c35f1"
format_stamp: "Formatted at 2026-08-12 14:21:43 on dist-test-slave-5k54"
server_key: "1145d83d0e984fa172c8219676b0f3a7"
server_key_iv: "fa1b271bfb75304a3cc588d4f0e9bcc3"
server_key_version: "encryptionkey@0"
I20260812 14:21:43.297369 29987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:21:43.298141 30002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:43.298314 29987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:21:43.298424 29987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "846b09b7734043ff872c79d6710c35f1"
format_stamp: "Formatted at 2026-08-12 14:21:43 on dist-test-slave-5k54"
server_key: "1145d83d0e984fa172c8219676b0f3a7"
server_key_iv: "fa1b271bfb75304a3cc588d4f0e9bcc3"
server_key_version: "encryptionkey@0"
I20260812 14:21:43.298499 29987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:43.309469 29987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:43.311168 29987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:43.311321 29987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:43.311565 29987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:43.311866 29987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:43.311924 29987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:43.311964 29987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:43.311992 29987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:43.317826 29987 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:37091
I20260812 14:21:43.317878 30141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:37091 every 8 connection(s)
I20260812 14:21:43.318159 29987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:21:43.323040 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 29987
I20260812 14:21:43.323154 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:21:43.323348 8532 external_mini_cluster.cc:1428] Setting key 3b6ff21724b2658b58e20bbc5c9ad98d
Aug 12 14:21:43 dist-test-slave-5k54 krb5kdc[29546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544503, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:21:43.327059 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.318767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:33315 (local address 127.8.85.62:43571)
0812 14:21:43.318813 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:43.318815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:43.319026 (+ 211us) server_negotiation.cc:407] Connection header received
0812 14:21:43.319582 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:43.319584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:43.319596 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:43.319658 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:43.320560 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.321060 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.322014 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.322202 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.322262 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.324503 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:43.324505 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:43.324505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:43.324512 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:43.325966 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:43.326323 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:43.326324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:43.326324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:43.326388 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:43.326563 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:43.326563 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:43.326564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:43.326664 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:43.326713 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:43.326936 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 14:21:43.326994 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:21:43.327266 30144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.318720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:43571 (local address 127.8.85.3:33315)
0812 14:21:43.318888 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:43.318930 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:43.319412 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:43.319735 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:43.319738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:43.320279 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:43.320467 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.320472 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.321143 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.321144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.321930 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.321932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.322285 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.322286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.322396 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.322827 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:21:43.322844 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:43.324417 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:43.326044 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:43.326050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:43.326051 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:43.326256 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:43.326430 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:43.326431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:43.326431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:43.326511 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:43.326725 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:43.326729 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:43.326833 (+ 104us) client_negotiation.cc:769] Sending connection context
0812 14:21:43.326913 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 14:21:43.327029 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260812 14:21:43.327798 30142 heartbeater.cc:347] Connected to a master server at 127.8.85.62:43571
I20260812 14:21:43.327881 30142 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:43.328059 30142 heartbeater.cc:511] Master 127.8.85.62:43571 requested a full tablet report, sending...
I20260812 14:21:43.328425 29599 ts_manager.cc:194] Registered new tserver with Master: 846b09b7734043ff872c79d6710c35f1 (127.8.85.3:37091)
I20260812 14:21:43.328799 29599 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:33315
I20260812 14:21:43.335034 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:21:43.342706 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.336484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40512 (local address 127.8.85.62:43571)
0812 14:21:43.336546 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:43.336549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:43.336665 (+ 116us) server_negotiation.cc:407] Connection header received
0812 14:21:43.336732 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:43.336733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:43.336752 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:43.336817 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:43.337434 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.337908 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.338651 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.338845 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.338926 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.340299 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:43.340300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:43.340301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:43.340308 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:43.341455 (+ 1147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:43.341882 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:43.341884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:43.341884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:43.341955 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:43.342161 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:43.342161 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:43.342161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:43.342287 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:43.342413 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:43.342518 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 14:21:43.342598 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:21:43.344262 29599 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:21:43.345758 29599 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:21:43.352339 30154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.348080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:40511 (local address 127.0.0.1:50036)
0812 14:21:43.348456 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:43.348466 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:43.348507 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:43.348950 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:43.348952 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:43.348954 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:43.348999 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.349001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.349895 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.349896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.351214 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.351217 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.351987 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.351988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.352107 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.352123 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 14:21:43.352198 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 14:21:43.352256 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":38,"spinlock_wait_cycles":10624,"thread_start_us":101,"threads_started":1}
I20260812 14:21:43.352519 30161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.348500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45968 (local address 127.8.85.2:45867)
0812 14:21:43.348693 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:43.348697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:43.348708 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:21:43.348746 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:43.348748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:43.348764 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:43.348847 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:43.349072 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.350118 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.351252 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.352010 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.352082 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.352140 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:43.352387 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 14:21:43.352429 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":89,"threads_started":1}
I20260812 14:21:43.352535 30158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.348183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60544 (local address 127.8.85.3:37091)
0812 14:21:43.348383 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:43.348388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:43.348398 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:21:43.348496 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:43.348499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:43.348517 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:43.348594 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:43.348875 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.350050 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.351394 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.352093 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.352161 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.352209 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:43.352296 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 14:21:43.352340 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":86,"threads_started":1}
I20260812 14:21:43.352591 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.348279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50036 (local address 127.8.85.1:40511)
0812 14:21:43.348766 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:43.348771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:43.348781 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:21:43.348820 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:43.348822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:43.348848 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:43.349071 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:43.349142 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.349785 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.351316 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.351884 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.351983 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.352048 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:43.352470 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 14:21:43.352513 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":95,"threads_started":1}
I20260812 14:21:43.352906 30156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.348079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:37091 (local address 127.0.0.1:60544)
0812 14:21:43.348243 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:43.348287 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:43.348367 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:43.348624 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:43.348692 (+ 68us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:43.348695 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:43.348779 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.348782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.350154 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.350155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.351310 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.351312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.352170 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.352171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.352230 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.352234 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:21:43.352285 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:21:43.352323 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":65,"thread_start_us":65,"threads_started":1}
I20260812 14:21:43.353104 30159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.348281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:45867 (local address 127.0.0.1:45968)
0812 14:21:43.348598 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:43.348608 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:43.348646 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:43.348944 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:43.348945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:43.348947 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:43.348991 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.348992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.350195 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.350196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.351148 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.351150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.352179 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.352180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.352245 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.352249 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:21:43.352299 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 14:21:43.352345 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":4,"thread_start_us":66,"threads_started":1}
I20260812 14:21:43.353351 29747 tablet_service.cc:1467] Processing CreateTablet for tablet f8520bdaf3f240abb418563871c98b6f (DEFAULT_TABLE table=test-table [id=c36e99570d70433e92188edfef584f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:43.353681 29747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8520bdaf3f240abb418563871c98b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:43.353612 30060 tablet_service.cc:1467] Processing CreateTablet for tablet f8520bdaf3f240abb418563871c98b6f (DEFAULT_TABLE table=test-table [id=c36e99570d70433e92188edfef584f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:43.353721 29910 tablet_service.cc:1467] Processing CreateTablet for tablet f8520bdaf3f240abb418563871c98b6f (DEFAULT_TABLE table=test-table [id=c36e99570d70433e92188edfef584f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:43.353935 30060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8520bdaf3f240abb418563871c98b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:43.354012 29910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8520bdaf3f240abb418563871c98b6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:43.356412 30162 tablet_bootstrap.cc:492] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c: Bootstrap starting.
I20260812 14:21:43.357005 30162 tablet_bootstrap.cc:654] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:43.357311 30163 tablet_bootstrap.cc:492] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7: Bootstrap starting.
I20260812 14:21:43.357625 30164 tablet_bootstrap.cc:492] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1: Bootstrap starting.
I20260812 14:21:43.357681 30162 log.cc:824] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:43.358150 30163 tablet_bootstrap.cc:654] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:43.358340 30164 tablet_bootstrap.cc:654] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:43.358675 30162 tablet_bootstrap.cc:492] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c: No bootstrap required, opened a new log
I20260812 14:21:43.358754 30162 ts_tablet_manager.cc:1397] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:21:43.358764 30163 log.cc:824] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:43.358880 30164 log.cc:824] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:43.359653 30164 tablet_bootstrap.cc:492] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1: No bootstrap required, opened a new log
I20260812 14:21:43.359733 30164 ts_tablet_manager.cc:1397] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:21:43.359989 30163 tablet_bootstrap.cc:492] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7: No bootstrap required, opened a new log
I20260812 14:21:43.360061 30163 ts_tablet_manager.cc:1397] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:21:43.360713 30162 raft_consensus.cc:348] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "846b09b7734043ff872c79d6710c35f1" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37091 } } peers { permanent_uuid: "40d60503e29d442fa1f4cfb66d0f2b9c" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 40511 } } peers { permanent_uuid: "92dbc197994943d4a36c4f6e50617ba7" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 45867 } }
I20260812 14:21:43.360903 30162 raft_consensus.cc:374] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:43.360942 30162 raft_consensus.cc:729] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40d60503e29d442fa1f4cfb66d0f2b9c, State: Initialized, Role: FOLLOWER
I20260812 14:21:43.361084 30162 consensus_queue.cc:260] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "846b09b7734043ff872c79d6710c35f1" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37091 } } peers { permanent_uuid: "40d60503e29d442fa1f4cfb66d0f2b9c" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 40511 } } peers { permanent_uuid: "92dbc197994943d4a36c4f6e50617ba7" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 45867 } }
I20260812 14:21:43.361369 30162 ts_tablet_manager.cc:1428] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:21:43.361488 29829 heartbeater.cc:503] Master 127.8.85.62:43571 was elected leader, sending a full tablet report...
I20260812 14:21:43.361492 30164 raft_consensus.cc:348] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "846b09b7734043ff872c79d6710c35f1" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37091 } } peers { permanent_uuid: "40d60503e29d442fa1f4cfb66d0f2b9c" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 40511 } } peers { permanent_uuid: "92dbc197994943d4a36c4f6e50617ba7" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 45867 } }
I20260812 14:21:43.361685 30164 raft_consensus.cc:374] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:43.361737 30164 raft_consensus.cc:729] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 846b09b7734043ff872c79d6710c35f1, State: Initialized, Role: FOLLOWER
I20260812 14:21:43.361873 30164 consensus_queue.cc:260] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "846b09b7734043ff872c79d6710c35f1" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37091 } } peers { permanent_uuid: "40d60503e29d442fa1f4cfb66d0f2b9c" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 40511 } } peers { permanent_uuid: "92dbc197994943d4a36c4f6e50617ba7" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 45867 } }
I20260812 14:21:43.362182 30164 ts_tablet_manager.cc:1428] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 14:21:43.362119 30163 raft_consensus.cc:348] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "846b09b7734043ff872c79d6710c35f1" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37091 } } peers { permanent_uuid: "40d60503e29d442fa1f4cfb66d0f2b9c" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 40511 } } peers { permanent_uuid: "92dbc197994943d4a36c4f6e50617ba7" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 45867 } }
I20260812 14:21:43.362306 30163 raft_consensus.cc:374] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:43.362360 30163 raft_consensus.cc:729] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92dbc197994943d4a36c4f6e50617ba7, State: Initialized, Role: FOLLOWER
I20260812 14:21:43.362470 30163 consensus_queue.cc:260] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "846b09b7734043ff872c79d6710c35f1" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37091 } } peers { permanent_uuid: "40d60503e29d442fa1f4cfb66d0f2b9c" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 40511 } } peers { permanent_uuid: "92dbc197994943d4a36c4f6e50617ba7" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 45867 } }
I20260812 14:21:43.362763 30163 ts_tablet_manager.cc:1428] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:21:43.362771 29981 heartbeater.cc:503] Master 127.8.85.62:43571 was elected leader, sending a full tablet report...
I20260812 14:21:43.362888 30142 heartbeater.cc:503] Master 127.8.85.62:43571 was elected leader, sending a full tablet report...
W20260812 14:21:43.418525 29982 tablet.cc:2369] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:21:43.515655 29830 tablet.cc:2369] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:21:43.534649 30169 raft_consensus.cc:482] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:21:43.534794 30169 raft_consensus.cc:504] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "846b09b7734043ff872c79d6710c35f1" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37091 } } peers { permanent_uuid: "40d60503e29d442fa1f4cfb66d0f2b9c" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 40511 } } peers { permanent_uuid: "92dbc197994943d4a36c4f6e50617ba7" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 45867 } }
I20260812 14:21:43.535372 30169 leader_election.cc:288] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40d60503e29d442fa1f4cfb66d0f2b9c (127.8.85.1:40511), 92dbc197994943d4a36c4f6e50617ba7 (127.8.85.2:45867)
I20260812 14:21:43.538429 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.535629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:52103 (local address 127.8.85.1:40511)
0812 14:21:43.535679 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:43.535681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:43.535694 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:21:43.535795 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:43.535796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:43.535823 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:43.535891 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:43.536060 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.536637 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.537678 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.538062 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.538114 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.538123 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:43.538314 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 14:21:43.538356 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:21:43.538445 30144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.535577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:40511 (local address 127.8.85.3:52103)
0812 14:21:43.535635 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:43.535650 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:43.535697 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:43.535902 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:43.535904 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:43.535905 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:43.535984 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.535986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.536720 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.536721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.537585 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.537587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.538138 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.538139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.538218 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.538221 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:21:43.538272 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:21:43.538356 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":49}
I20260812 14:21:43.538497 30161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.535714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:39713 (local address 127.8.85.2:45867)
0812 14:21:43.535759 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:43.535761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:43.535884 (+ 123us) server_negotiation.cc:407] Connection header received
0812 14:21:43.535918 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:43.535919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:43.535931 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:43.535994 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:43.536192 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.536766 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.537715 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.538086 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.538177 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.538186 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:43.538367 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 14:21:43.538416 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:21:43.538728 30171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.535597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:45867 (local address 127.8.85.3:39713)
0812 14:21:43.535782 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:43.535793 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:43.535872 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:43.536003 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:43.536003 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:43.536005 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:43.536093 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.536094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.536839 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.536839 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.537596 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.537598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.538191 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.538192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.538244 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.538248 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:21:43.538340 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 14:21:43.538383 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":1,"thread_start_us":121,"threads_started":1}
I20260812 14:21:43.538878 29775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8520bdaf3f240abb418563871c98b6f" candidate_uuid: "846b09b7734043ff872c79d6710c35f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40d60503e29d442fa1f4cfb66d0f2b9c" is_pre_election: true
I20260812 14:21:43.539052 29775 raft_consensus.cc:2454] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 846b09b7734043ff872c79d6710c35f1 in term 0.
I20260812 14:21:43.539296 30005 leader_election.cc:302] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40d60503e29d442fa1f4cfb66d0f2b9c, 846b09b7734043ff872c79d6710c35f1; no voters:
I20260812 14:21:43.539264 29932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8520bdaf3f240abb418563871c98b6f" candidate_uuid: "846b09b7734043ff872c79d6710c35f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92dbc197994943d4a36c4f6e50617ba7" is_pre_election: true
I20260812 14:21:43.539453 29932 raft_consensus.cc:2454] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 846b09b7734043ff872c79d6710c35f1 in term 0.
I20260812 14:21:43.539455 30169 raft_consensus.cc:2781] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:21:43.539521 30169 raft_consensus.cc:482] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:21:43.539546 30169 raft_consensus.cc:3037] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:43.540278 30169 raft_consensus.cc:504] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "846b09b7734043ff872c79d6710c35f1" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37091 } } peers { permanent_uuid: "40d60503e29d442fa1f4cfb66d0f2b9c" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 40511 } } peers { permanent_uuid: "92dbc197994943d4a36c4f6e50617ba7" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 45867 } }
I20260812 14:21:43.540438 30169 leader_election.cc:288] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [CANDIDATE]: Term 1 election: Requested vote from peers 40d60503e29d442fa1f4cfb66d0f2b9c (127.8.85.1:40511), 92dbc197994943d4a36c4f6e50617ba7 (127.8.85.2:45867)
I20260812 14:21:43.540621 29932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8520bdaf3f240abb418563871c98b6f" candidate_uuid: "846b09b7734043ff872c79d6710c35f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92dbc197994943d4a36c4f6e50617ba7"
I20260812 14:21:43.540625 29775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8520bdaf3f240abb418563871c98b6f" candidate_uuid: "846b09b7734043ff872c79d6710c35f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40d60503e29d442fa1f4cfb66d0f2b9c"
I20260812 14:21:43.540700 29775 raft_consensus.cc:3037] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:43.540727 29932 raft_consensus.cc:3037] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:43.541429 29932 raft_consensus.cc:2454] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 846b09b7734043ff872c79d6710c35f1 in term 1.
I20260812 14:21:43.541438 29775 raft_consensus.cc:2454] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 846b09b7734043ff872c79d6710c35f1 in term 1.
I20260812 14:21:43.541611 30005 leader_election.cc:302] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40d60503e29d442fa1f4cfb66d0f2b9c, 846b09b7734043ff872c79d6710c35f1; no voters:
I20260812 14:21:43.541718 30169 raft_consensus.cc:2781] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:43.541921 30169 raft_consensus.cc:686] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [term 1 LEADER]: Becoming Leader. State: Replica: 846b09b7734043ff872c79d6710c35f1, State: Running, Role: LEADER
I20260812 14:21:43.542026 30169 consensus_queue.cc:237] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "846b09b7734043ff872c79d6710c35f1" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37091 } } peers { permanent_uuid: "40d60503e29d442fa1f4cfb66d0f2b9c" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 40511 } } peers { permanent_uuid: "92dbc197994943d4a36c4f6e50617ba7" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 45867 } }
I20260812 14:21:43.542907 29598 catalog_manager.cc:5495] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 846b09b7734043ff872c79d6710c35f1 (127.8.85.3). New cstate: current_term: 1 leader_uuid: "846b09b7734043ff872c79d6710c35f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "846b09b7734043ff872c79d6710c35f1" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 37091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40d60503e29d442fa1f4cfb66d0f2b9c" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 40511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92dbc197994943d4a36c4f6e50617ba7" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 45867 } health_report { overall_health: UNKNOWN } } }
W20260812 14:21:43.568691 30143 tablet.cc:2369] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:21:43.580061 30158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.577558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60560 (local address 127.8.85.3:37091)
0812 14:21:43.577620 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:43.577622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:43.577633 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:21:43.577676 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:43.577677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:43.577699 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:43.577790 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:21:43.577949 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.578425 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.579184 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.579403 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.579529 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.579698 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:21:43.579800 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:21:43.579919 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:43.579952 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 14:21:43.579961 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:21:43.581838 29932 raft_consensus.cc:1261] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7 [term 1 FOLLOWER]: Refusing update from remote peer 846b09b7734043ff872c79d6710c35f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:21:43.581845 29775 raft_consensus.cc:1261] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c [term 1 FOLLOWER]: Refusing update from remote peer 846b09b7734043ff872c79d6710c35f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:21:43.582250 30172 consensus_queue.cc:1035] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92dbc197994943d4a36c4f6e50617ba7" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 45867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:21:43.582396 30169 consensus_queue.cc:1035] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40d60503e29d442fa1f4cfb66d0f2b9c" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 40511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:21:43.585496 30178 mvcc.cc:205] Tried to move back new op lower bound from 7317686286668509184 to 7317686286508867584. Current Snapshot: MvccSnapshot[applied={T|T < 7317686286668509184}]
I20260812 14:21:43.585948 30180 mvcc.cc:205] Tried to move back new op lower bound from 7317686286668509184 to 7317686286508867584. Current Snapshot: MvccSnapshot[applied={T|T < 7317686286668509184}]
I20260812 14:21:43.587399 29598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:21:43.587522 29598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:21:43.588307 29598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c625b5cf9c9746ad839c332a9982a1a7: Sending DeleteTablet for 3 replicas of tablet f8520bdaf3f240abb418563871c98b6f
I20260812 14:21:43.588690 29747 tablet_service.cc:1514] Processing DeleteTablet for tablet f8520bdaf3f240abb418563871c98b6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:43 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:50036
I20260812 14:21:43.588704 29910 tablet_service.cc:1514] Processing DeleteTablet for tablet f8520bdaf3f240abb418563871c98b6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:43 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:45968
I20260812 14:21:43.588714 30060 tablet_service.cc:1514] Processing DeleteTablet for tablet f8520bdaf3f240abb418563871c98b6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:43 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:60544
I20260812 14:21:43.588939 30189 tablet_replica.cc:330] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c: stopping tablet replica
I20260812 14:21:43.589176 30189 raft_consensus.cc:2229] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:21:43.589221 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 29669
I20260812 14:21:43.589327 30189 raft_consensus.cc:2258] T f8520bdaf3f240abb418563871c98b6f P 40d60503e29d442fa1f4cfb66d0f2b9c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 14:21:43.598153 29583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:40511 (error 108)
I20260812 14:21:43.598268 30190 tablet_replica.cc:330] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7: stopping tablet replica
I20260812 14:21:43.598382 30190 raft_consensus.cc:2229] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:21:43.598452 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 29836
I20260812 14:21:43.598495 30190 raft_consensus.cc:2258] T f8520bdaf3f240abb418563871c98b6f P 92dbc197994943d4a36c4f6e50617ba7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:43.598649 30191 tablet_replica.cc:330] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1: stopping tablet replica
I20260812 14:21:43.598735 30191 raft_consensus.cc:2229] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:21:43.598881 30191 raft_consensus.cc:2258] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:43.599651 30159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.599530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:40511 (local address 127.0.0.1:50040)
0812 14:21:43.599569 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:43.599609 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:40511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 14:21:43.599676 30191 ts_tablet_manager.cc:1905] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 14:21:43.599781 29583 catalog_manager.cc:4584] TS 40d60503e29d442fa1f4cfb66d0f2b9c (127.8.85.1:40511): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8520bdaf3f240abb418563871c98b6f: Network error: Client connection negotiation failed: client connection to 127.8.85.1:40511: connect: Connection refused (error 111)
I20260812 14:21:43.601946 30191 ts_tablet_manager.cc:1918] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:21:43.602052 30191 log.cc:1196] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/f8520bdaf3f240abb418563871c98b6f
I20260812 14:21:43.602344 30191 ts_tablet_manager.cc:1939] T f8520bdaf3f240abb418563871c98b6f P 846b09b7734043ff872c79d6710c35f1: Deleting consensus metadata
I20260812 14:21:43.603022 29585 catalog_manager.cc:4841] TS 846b09b7734043ff872c79d6710c35f1 (127.8.85.3:37091): tablet f8520bdaf3f240abb418563871c98b6f (table test-table [id=c36e99570d70433e92188edfef584f19]) successfully deleted
I20260812 14:21:43.604749 30159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.604625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:45867 (local address 127.0.0.1:45974)
0812 14:21:43.604684 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:43.604716 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.2:45867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 14:21:43.604924 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 29987
I20260812 14:21:43.611331 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 29562
2026-08-12T14:21:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2182 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T14:21:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:21:43Z Disabled control of system clock
I20260812 14:21:43.643764 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:34271
--webserver_interface=127.8.85.62
--webserver_port=0
--builtin_ntp_servers=127.8.85.20:34635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:34271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:21:43.725396 30197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:43.725625 30197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:43.725646 30197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:43.726744 30197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:43.726779 30197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:43.726792 30197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:43.726804 30197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:43.726817 30197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:43.728206 30197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:34635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:34271
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:34271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:43.728385 30197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:43.728619 30197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:43.733425 30205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:43.733378 30203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:43.733413 30202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:43.733572 30197 server_base.cc:1034] running on GCE node
I20260812 14:21:43.733883 30197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:43.734212 30197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:43.735363 30197 hybrid_clock.cc:648] HybridClock initialized: now 1786544503735357 us; error 35 us; skew 500 ppm
I20260812 14:21:43.736585 30197 webserver.cc:460] Webserver started at http://127.8.85.62:36025/ using document root <none> and password file <none>
I20260812 14:21:43.736847 30197 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:43.736917 30197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:43.737078 30197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:43.738147 30197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "d2a51c78461547b182eb731690a219a7"
format_stamp: "Formatted at 2026-08-12 14:21:43 on dist-test-slave-5k54"
server_key: "2e972b73b90b30e2311687f4a79e6d46"
server_key_iv: "ddde3d219cc92ec31b81ea13d55b5e72"
server_key_version: "encryptionkey@0"
I20260812 14:21:43.738516 30197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "d2a51c78461547b182eb731690a219a7"
format_stamp: "Formatted at 2026-08-12 14:21:43 on dist-test-slave-5k54"
server_key: "2e972b73b90b30e2311687f4a79e6d46"
server_key_iv: "ddde3d219cc92ec31b81ea13d55b5e72"
server_key_version: "encryptionkey@0"
I20260812 14:21:43.739936 30197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:21:43.740688 30212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:43.740860 30197 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:43.740960 30197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "d2a51c78461547b182eb731690a219a7"
format_stamp: "Formatted at 2026-08-12 14:21:43 on dist-test-slave-5k54"
server_key: "2e972b73b90b30e2311687f4a79e6d46"
server_key_iv: "ddde3d219cc92ec31b81ea13d55b5e72"
server_key_version: "encryptionkey@0"
I20260812 14:21:43.741055 30197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:43.765499 30197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:43.765833 30197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:43.766008 30197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:43.770227 30197 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:34271
I20260812 14:21:43.770262 30279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:34271 every 8 connection(s)
I20260812 14:21:43.770624 30197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:43.771687 30280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:43.774555 30280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7: Bootstrap starting.
I20260812 14:21:43.775334 30280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:43.775715 30280 log.cc:824] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:43.776680 30280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7: No bootstrap required, opened a new log
I20260812 14:21:43.778407 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 30197
I20260812 14:21:43.778523 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:21:43.778730 8532 external_mini_cluster.cc:1428] Setting key 04bd015993211ac81b3cadde8db4476c
I20260812 14:21:43.779369 30280 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a51c78461547b182eb731690a219a7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 34271 } }
I20260812 14:21:43.779584 30280 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:43.779629 30280 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2a51c78461547b182eb731690a219a7, State: Initialized, Role: FOLLOWER
I20260812 14:21:43.779767 30280 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a51c78461547b182eb731690a219a7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 34271 } }
I20260812 14:21:43.779844 30280 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:43.779872 30280 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:43.779920 30280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:43.780782 30280 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a51c78461547b182eb731690a219a7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 34271 } }
I20260812 14:21:43.780926 30280 leader_election.cc:302] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2a51c78461547b182eb731690a219a7; no voters:
I20260812 14:21:43.781107 30280 leader_election.cc:288] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:21:43.781282 30286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:43.781366 30280 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:43.781504 30286 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [term 1 LEADER]: Becoming Leader. State: Replica: d2a51c78461547b182eb731690a219a7, State: Running, Role: LEADER
I20260812 14:21:43.781648 30286 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a51c78461547b182eb731690a219a7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 34271 } }
I20260812 14:21:43.782361 30286 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2a51c78461547b182eb731690a219a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a51c78461547b182eb731690a219a7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 34271 } } }
I20260812 14:21:43.782537 30286 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:43.782557 30288 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2a51c78461547b182eb731690a219a7. Latest consensus state: current_term: 1 leader_uuid: "d2a51c78461547b182eb731690a219a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2a51c78461547b182eb731690a219a7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 34271 } } }
I20260812 14:21:43.782663 30288 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:43.782884 30295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:43.783401 30295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:43.785774 30295 catalog_manager.cc:1347] Generated new cluster ID: abc4371ec6474384a99e2ae011b73b6b
I20260812 14:21:43.785832 30295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:21:43.788264 30284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.779346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46496 (local address 127.8.85.62:34271)
0812 14:21:43.781183 (+ 1837us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:43.781194 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:43.781231 (+ 37us) server_negotiation.cc:407] Connection header received
0812 14:21:43.781359 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:43.781366 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:43.781472 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:43.782952 (+ 1480us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:43.783232 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.784789 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.785957 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.786549 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.786976 (+ 427us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.787403 (+ 427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:43.787406 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:43.787407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:43.787630 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:43.787667 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:43.787680 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:43.787946 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:43.787978 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 14:21:43.788054 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1755,"thread_start_us":125,"threads_started":1}
I20260812 14:21:43.804234 30295 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:21:43.804968 30295 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:43.812608 30295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7: Generated new TSK 0
I20260812 14:21:43.812801 30295 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:21:43.820644 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:34271
--builtin_ntp_servers=127.8.85.20:34635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:21:43.904964 30307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:43.905198 30307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:43.905233 30307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:43.906385 30307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:43.906450 30307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:43.906499 30307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:21:43.907859 30307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:34635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:34271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:43.908115 30307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:43.908404 30307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:43.914078 30313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:43.914070 30312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:43.914070 30315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:43.914469 30307 server_base.cc:1034] running on GCE node
I20260812 14:21:43.914687 30307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:43.914883 30307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:43.916038 30307 hybrid_clock.cc:648] HybridClock initialized: now 1786544503916027 us; error 27 us; skew 500 ppm
I20260812 14:21:43.917294 30307 webserver.cc:460] Webserver started at http://127.8.85.1:34573/ using document root <none> and password file <none>
I20260812 14:21:43.917555 30307 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:43.917604 30307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:43.917783 30307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:43.918848 30307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "eea3c8f431bd4b82be0f155b05811a3f"
format_stamp: "Formatted at 2026-08-12 14:21:43 on dist-test-slave-5k54"
server_key: "ed23611c1d31c51aa5149426c6ec9f0b"
server_key_iv: "bb436e2c13120b93ce2b62582e4e2653"
server_key_version: "encryptionkey@0"
I20260812 14:21:43.919219 30307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "eea3c8f431bd4b82be0f155b05811a3f"
format_stamp: "Formatted at 2026-08-12 14:21:43 on dist-test-slave-5k54"
server_key: "ed23611c1d31c51aa5149426c6ec9f0b"
server_key_iv: "bb436e2c13120b93ce2b62582e4e2653"
server_key_version: "encryptionkey@0"
I20260812 14:21:43.920636 30307 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:21:43.921355 30323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:43.921523 30307 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:21:43.921633 30307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "eea3c8f431bd4b82be0f155b05811a3f"
format_stamp: "Formatted at 2026-08-12 14:21:43 on dist-test-slave-5k54"
server_key: "ed23611c1d31c51aa5149426c6ec9f0b"
server_key_iv: "bb436e2c13120b93ce2b62582e4e2653"
server_key_version: "encryptionkey@0"
I20260812 14:21:43.921710 30307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:43.943988 30307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:43.944293 30307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:43.944444 30307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:43.944690 30307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:43.945052 30307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:43.945111 30307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:43.945154 30307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:43.945179 30307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:43.950863 30307 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:45187
I20260812 14:21:43.950908 30459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:45187 every 8 connection(s)
I20260812 14:21:43.951283 30307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:21:43.956802 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 30307
I20260812 14:21:43.956851 30284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.952051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:35133 (local address 127.8.85.62:34271)
0812 14:21:43.952125 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:43.952127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:43.952366 (+ 239us) server_negotiation.cc:407] Connection header received
0812 14:21:43.952937 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:43.952938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:43.952953 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:43.953010 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:43.953698 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.954214 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.955463 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.955692 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.955769 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.956557 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:43.956559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:43.956559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:43.956564 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:43.956580 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:43.956585 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:43.956629 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:43.956733 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 14:21:43.956769 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:21:43.956939 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:21:43.957103 30464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:43.952040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:34271 (local address 127.8.85.1:35133)
0812 14:21:43.952240 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:43.952275 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:43.952819 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:43.953086 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:43.953089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:43.953097 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:43.953600 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.953604 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.954321 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.954322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.955308 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:43.955311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:43.955827 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:43.955828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:43.955950 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:43.956412 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:21:43.956413 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:43.956440 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:21:43.956462 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:43.956640 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:43.956645 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:43.956649 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:21:43.956734 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 14:21:43.956884 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260812 14:21:43.957127 8532 external_mini_cluster.cc:1428] Setting key c7094b36371bef308f3ebe0cecc6b521
I20260812 14:21:43.957732 30460 heartbeater.cc:347] Connected to a master server at 127.8.85.62:34271
I20260812 14:21:43.957845 30460 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:43.958070 30460 heartbeater.cc:511] Master 127.8.85.62:34271 requested a full tablet report, sending...
I20260812 14:21:43.958345 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:34271
--builtin_ntp_servers=127.8.85.20:34635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 14:21:43.958647 30234 ts_manager.cc:194] Registered new tserver with Master: eea3c8f431bd4b82be0f155b05811a3f (127.8.85.1:45187)
I20260812 14:21:43.959275 30234 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.85.1:35133
W20260812 14:21:44.041085 30465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:44.041297 30465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:44.041319 30465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:44.042383 30465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:44.042414 30465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:44.042438 30465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:21:44.043817 30465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:34635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:34271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:44.044086 30465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:44.044338 30465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:44.049230 30471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:44.049290 30465 server_base.cc:1034] running on GCE node
W20260812 14:21:44.049237 30470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:44.049249 30473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:44.049643 30465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:44.049870 30465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:44.051026 30465 hybrid_clock.cc:648] HybridClock initialized: now 1786544504051015 us; error 23 us; skew 500 ppm
I20260812 14:21:44.052261 30465 webserver.cc:460] Webserver started at http://127.8.85.2:43225/ using document root <none> and password file <none>
I20260812 14:21:44.052515 30465 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:44.052582 30465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:44.052732 30465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:44.053743 30465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "534b3bdf0b364f1789738520adfceb2b"
format_stamp: "Formatted at 2026-08-12 14:21:44 on dist-test-slave-5k54"
server_key: "37fd2f95181a1012f9d3d26a14ba48a1"
server_key_iv: "5f8777d4f598c8efa62161d5e913202e"
server_key_version: "encryptionkey@0"
I20260812 14:21:44.054097 30465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "534b3bdf0b364f1789738520adfceb2b"
format_stamp: "Formatted at 2026-08-12 14:21:44 on dist-test-slave-5k54"
server_key: "37fd2f95181a1012f9d3d26a14ba48a1"
server_key_iv: "5f8777d4f598c8efa62161d5e913202e"
server_key_version: "encryptionkey@0"
I20260812 14:21:44.055344 30465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:21:44.056061 30482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:44.056241 30465 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:44.056334 30465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "534b3bdf0b364f1789738520adfceb2b"
format_stamp: "Formatted at 2026-08-12 14:21:44 on dist-test-slave-5k54"
server_key: "37fd2f95181a1012f9d3d26a14ba48a1"
server_key_iv: "5f8777d4f598c8efa62161d5e913202e"
server_key_version: "encryptionkey@0"
I20260812 14:21:44.056411 30465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:44.065547 30465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:44.065814 30465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:44.065958 30465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:44.066180 30465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:44.066486 30465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:44.066540 30465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:44.066578 30465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:44.066604 30465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:44.072623 30465 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:43041
I20260812 14:21:44.072672 30617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:43041 every 8 connection(s)
I20260812 14:21:44.072984 30465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:21:44.074127 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 30465
I20260812 14:21:44.074244 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:21:44.074450 8532 external_mini_cluster.cc:1428] Setting key 1dd705bf32303a38d3f9f8403e90628b
I20260812 14:21:44.075616 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:34271
--builtin_ntp_servers=127.8.85.20:34635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 14:21:44.078727 30284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.073747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:51073 (local address 127.8.85.62:34271)
0812 14:21:44.073799 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:44.073800 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:44.073993 (+ 193us) server_negotiation.cc:407] Connection header received
0812 14:21:44.074539 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:44.074541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:44.074556 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:44.074644 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:44.075567 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.076319 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.077276 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.077495 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.077581 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.078436 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:44.078437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:44.078437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:44.078442 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:44.078457 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:44.078461 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:44.078511 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:44.078615 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 14:21:44.078645 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:21:44.079032 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.073672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:34271 (local address 127.8.85.2:51073)
0812 14:21:44.073845 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:44.073902 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:44.074430 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:44.074729 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:44.074733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:44.074743 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:44.075467 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:44.075480 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.076409 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.076409 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:44.077182 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:44.077184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.077587 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.077588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:44.077724 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.078283 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:21:44.078284 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:44.078312 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:21:44.078335 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:44.078522 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:44.078527 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:44.078531 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:21:44.078615 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 14:21:44.078713 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260812 14:21:44.079531 30618 heartbeater.cc:347] Connected to a master server at 127.8.85.62:34271
I20260812 14:21:44.079619 30618 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:44.079807 30618 heartbeater.cc:511] Master 127.8.85.62:34271 requested a full tablet report, sending...
I20260812 14:21:44.080188 30234 ts_manager.cc:194] Registered new tserver with Master: 534b3bdf0b364f1789738520adfceb2b (127.8.85.2:43041)
I20260812 14:21:44.080564 30234 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.85.2:51073
W20260812 14:21:44.160175 30621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:44.160398 30621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:44.160435 30621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:44.161582 30621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:44.161648 30621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:44.161689 30621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:21:44.163070 30621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:34635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:34271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:44.163331 30621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:44.163662 30621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:44.168627 30626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:44.168684 30627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:44.168649 30629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:44.168888 30621 server_base.cc:1034] running on GCE node
I20260812 14:21:44.169075 30621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:44.169265 30621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:44.170432 30621 hybrid_clock.cc:648] HybridClock initialized: now 1786544504170425 us; error 35 us; skew 500 ppm
I20260812 14:21:44.171813 30621 webserver.cc:460] Webserver started at http://127.8.85.3:44083/ using document root <none> and password file <none>
I20260812 14:21:44.172077 30621 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:44.172149 30621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:44.172307 30621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:44.173377 30621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "8098227618cb47e6ba40fe20b0a04d13"
format_stamp: "Formatted at 2026-08-12 14:21:44 on dist-test-slave-5k54"
server_key: "3d72c55233a51f082b51086906331c3a"
server_key_iv: "1032fb3102f92b55ff374741a36ab4c5"
server_key_version: "encryptionkey@0"
I20260812 14:21:44.173741 30621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "8098227618cb47e6ba40fe20b0a04d13"
format_stamp: "Formatted at 2026-08-12 14:21:44 on dist-test-slave-5k54"
server_key: "3d72c55233a51f082b51086906331c3a"
server_key_iv: "1032fb3102f92b55ff374741a36ab4c5"
server_key_version: "encryptionkey@0"
I20260812 14:21:44.175100 30621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:21:44.175940 30635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:44.176113 30621 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:44.176219 30621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "8098227618cb47e6ba40fe20b0a04d13"
format_stamp: "Formatted at 2026-08-12 14:21:44 on dist-test-slave-5k54"
server_key: "3d72c55233a51f082b51086906331c3a"
server_key_iv: "1032fb3102f92b55ff374741a36ab4c5"
server_key_version: "encryptionkey@0"
I20260812 14:21:44.176292 30621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:44.188865 30621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:44.189157 30621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:44.189307 30621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:44.189541 30621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:44.189860 30621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:44.189916 30621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:44.189957 30621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:44.189985 30621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:44.195936 30621 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:38065
I20260812 14:21:44.195990 30773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:38065 every 8 connection(s)
I20260812 14:21:44.196350 30621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:21:44.201576 30284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.197142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:36715 (local address 127.8.85.62:34271)
0812 14:21:44.197206 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:44.197208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:44.197452 (+ 244us) server_negotiation.cc:407] Connection header received
0812 14:21:44.197942 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:44.197943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:44.197954 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:44.198006 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:44.198637 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.199163 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.200211 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.200393 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.200446 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.201267 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:44.201268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:44.201268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:44.201272 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:44.201284 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:44.201288 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:44.201329 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:44.201421 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 14:21:44.201470 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:21:44.201659 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 30621
I20260812 14:21:44.201668 30776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.197133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:34271 (local address 127.8.85.3:36715)
0812 14:21:44.197317 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:44.197351 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:44.197846 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:44.198061 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:44.198064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:44.198073 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:44.198567 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:44.198572 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.199249 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.199250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:44.200137 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:44.200140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.200474 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.200475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:44.200582 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.201139 (+ 557us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:21:44.201140 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:44.201169 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:21:44.201191 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:44.201344 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:44.201347 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:44.201350 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:21:44.201409 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 14:21:44.201485 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260812 14:21:44.201763 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:21:44.201964 8532 external_mini_cluster.cc:1428] Setting key 1758ef78198f3522017b22432c193610
I20260812 14:21:44.202369 30774 heartbeater.cc:347] Connected to a master server at 127.8.85.62:34271
I20260812 14:21:44.202489 30774 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:44.202689 30774 heartbeater.cc:511] Master 127.8.85.62:34271 requested a full tablet report, sending...
I20260812 14:21:44.203119 30234 ts_manager.cc:194] Registered new tserver with Master: 8098227618cb47e6ba40fe20b0a04d13 (127.8.85.3:38065)
I20260812 14:21:44.203568 30234 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.85.3:36715
I20260812 14:21:44.213930 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:21:44.218318 30284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.215356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46506 (local address 127.8.85.62:34271)
0812 14:21:44.215456 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:44.215459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:44.215557 (+ 98us) server_negotiation.cc:407] Connection header received
0812 14:21:44.215606 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:44.215607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:44.215617 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:44.215668 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:44.215836 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.216373 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.217176 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.217350 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.217399 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.218034 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:44.218035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:44.218035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:44.218039 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:44.218053 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:44.218057 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:44.218134 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:44.218253 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 14:21:44.218263 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 14:21:44.220103 30234 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:21:44.221679 30234 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:21:44.228102 30786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.223884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:38065 (local address 127.0.0.1:35434)
0812 14:21:44.224259 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:44.224299 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:44.224402 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:44.224682 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:44.224685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:44.224688 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:44.224749 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:44.224752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.225742 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.225743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:44.226883 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:44.226887 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.227789 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.227790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:44.227867 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.227879 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 14:21:44.227953 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 14:21:44.228010 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":77,"threads_started":1}
I20260812 14:21:44.228106 30784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.223977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35434 (local address 127.8.85.3:38065)
0812 14:21:44.224168 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:44.224174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:44.224368 (+ 194us) server_negotiation.cc:407] Connection header received
0812 14:21:44.224506 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:44.224508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:44.224525 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:44.224580 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:44.224860 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.225662 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.227007 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.227693 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.227769 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.227854 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:44.227980 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 14:21:44.228022 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260812 14:21:44.228384 30785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.224087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52094 (local address 127.8.85.1:45187)
0812 14:21:44.224234 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:44.224239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:44.224493 (+ 254us) server_negotiation.cc:407] Connection header received
0812 14:21:44.224568 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:44.224570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:44.224588 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:44.224662 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:44.224902 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.226055 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.227113 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.227779 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.227849 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.227943 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:44.228274 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 14:21:44.228311 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":82,"threads_started":1}
I20260812 14:21:44.228199 30783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.223884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:45187 (local address 127.0.0.1:52094)
0812 14:21:44.224291 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:44.224303 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:44.224396 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:44.224739 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:44.224740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:44.224743 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:44.224818 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:44.224820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.226140 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.226141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:44.227024 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:44.227026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.227989 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.227990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:44.228040 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.228042 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:21:44.228089 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 14:21:44.228123 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":20,"thread_start_us":96,"threads_started":1}
I20260812 14:21:44.228623 30782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.224129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:43041 (local address 127.0.0.1:49756)
0812 14:21:44.224352 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:44.224361 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:44.224403 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:44.224748 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:44.224749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:44.224750 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:44.224819 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:44.224822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.226119 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.226120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:44.227329 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:44.227332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.228367 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.228367 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:44.228442 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.228447 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:21:44.228505 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:21:44.228556 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":34,"thread_start_us":78,"threads_started":1}
I20260812 14:21:44.228904 30787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.224348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49756 (local address 127.8.85.2:43041)
0812 14:21:44.224554 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:44.224558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:44.224568 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:21:44.224604 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:44.224606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:44.224622 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:44.224689 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:44.224913 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.226016 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.227453 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.228181 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.228251 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.228335 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:44.228774 (+ 439us) server_negotiation.cc:299] Negotiation successful
0812 14:21:44.228817 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":92,"threads_started":1}
I20260812 14:21:44.229218 30381 tablet_service.cc:1467] Processing CreateTablet for tablet 0e551f0f009a491bab0f3db153e93af6 (DEFAULT_TABLE table=test-table [id=175dd2c37c6448dd9e67df93f09788e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:44.229640 30381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e551f0f009a491bab0f3db153e93af6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:44.229592 30534 tablet_service.cc:1467] Processing CreateTablet for tablet 0e551f0f009a491bab0f3db153e93af6 (DEFAULT_TABLE table=test-table [id=175dd2c37c6448dd9e67df93f09788e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:44.229933 30534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e551f0f009a491bab0f3db153e93af6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:44.231806 30690 tablet_service.cc:1467] Processing CreateTablet for tablet 0e551f0f009a491bab0f3db153e93af6 (DEFAULT_TABLE table=test-table [id=175dd2c37c6448dd9e67df93f09788e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:44.232214 30690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e551f0f009a491bab0f3db153e93af6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:44.233322 30789 tablet_bootstrap.cc:492] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b: Bootstrap starting.
I20260812 14:21:44.233568 30790 tablet_bootstrap.cc:492] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f: Bootstrap starting.
I20260812 14:21:44.234298 30790 tablet_bootstrap.cc:654] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:44.234640 30790 log.cc:824] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:44.235486 30790 tablet_bootstrap.cc:492] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f: No bootstrap required, opened a new log
I20260812 14:21:44.235585 30789 tablet_bootstrap.cc:654] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:44.235689 30790 ts_tablet_manager.cc:1397] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:21:44.235787 30792 tablet_bootstrap.cc:492] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13: Bootstrap starting.
I20260812 14:21:44.235945 30789 log.cc:824] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:44.236387 30792 tablet_bootstrap.cc:654] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:44.236879 30792 log.cc:824] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:44.236929 30789 tablet_bootstrap.cc:492] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b: No bootstrap required, opened a new log
I20260812 14:21:44.236990 30789 ts_tablet_manager.cc:1397] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 14:21:44.237704 30792 tablet_bootstrap.cc:492] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13: No bootstrap required, opened a new log
I20260812 14:21:44.237789 30792 ts_tablet_manager.cc:1397] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:21:44.238109 30790 raft_consensus.cc:348] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8098227618cb47e6ba40fe20b0a04d13" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 38065 } } peers { permanent_uuid: "534b3bdf0b364f1789738520adfceb2b" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43041 } } peers { permanent_uuid: "eea3c8f431bd4b82be0f155b05811a3f" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45187 } }
I20260812 14:21:44.238263 30790 raft_consensus.cc:374] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:44.238307 30790 raft_consensus.cc:729] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eea3c8f431bd4b82be0f155b05811a3f, State: Initialized, Role: FOLLOWER
I20260812 14:21:44.238422 30790 consensus_queue.cc:260] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8098227618cb47e6ba40fe20b0a04d13" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 38065 } } peers { permanent_uuid: "534b3bdf0b364f1789738520adfceb2b" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43041 } } peers { permanent_uuid: "eea3c8f431bd4b82be0f155b05811a3f" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45187 } }
I20260812 14:21:44.238693 30790 ts_tablet_manager.cc:1428] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:21:44.238827 30460 heartbeater.cc:503] Master 127.8.85.62:34271 was elected leader, sending a full tablet report...
I20260812 14:21:44.239599 30789 raft_consensus.cc:348] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8098227618cb47e6ba40fe20b0a04d13" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 38065 } } peers { permanent_uuid: "534b3bdf0b364f1789738520adfceb2b" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43041 } } peers { permanent_uuid: "eea3c8f431bd4b82be0f155b05811a3f" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45187 } }
I20260812 14:21:44.239761 30789 raft_consensus.cc:374] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:44.239805 30789 raft_consensus.cc:729] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 534b3bdf0b364f1789738520adfceb2b, State: Initialized, Role: FOLLOWER
I20260812 14:21:44.239910 30789 consensus_queue.cc:260] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8098227618cb47e6ba40fe20b0a04d13" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 38065 } } peers { permanent_uuid: "534b3bdf0b364f1789738520adfceb2b" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43041 } } peers { permanent_uuid: "eea3c8f431bd4b82be0f155b05811a3f" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45187 } }
I20260812 14:21:44.240177 30789 ts_tablet_manager.cc:1428] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:21:44.240224 30618 heartbeater.cc:503] Master 127.8.85.62:34271 was elected leader, sending a full tablet report...
I20260812 14:21:44.240391 30792 raft_consensus.cc:348] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8098227618cb47e6ba40fe20b0a04d13" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 38065 } } peers { permanent_uuid: "534b3bdf0b364f1789738520adfceb2b" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43041 } } peers { permanent_uuid: "eea3c8f431bd4b82be0f155b05811a3f" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45187 } }
I20260812 14:21:44.240569 30792 raft_consensus.cc:374] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:44.240622 30792 raft_consensus.cc:729] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8098227618cb47e6ba40fe20b0a04d13, State: Initialized, Role: FOLLOWER
I20260812 14:21:44.240747 30792 consensus_queue.cc:260] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8098227618cb47e6ba40fe20b0a04d13" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 38065 } } peers { permanent_uuid: "534b3bdf0b364f1789738520adfceb2b" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43041 } } peers { permanent_uuid: "eea3c8f431bd4b82be0f155b05811a3f" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45187 } }
I20260812 14:21:44.241048 30792 ts_tablet_manager.cc:1428] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 14:21:44.241092 30774 heartbeater.cc:503] Master 127.8.85.62:34271 was elected leader, sending a full tablet report...
I20260812 14:21:44.299023 30796 raft_consensus.cc:482] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:21:44.299154 30796 raft_consensus.cc:504] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8098227618cb47e6ba40fe20b0a04d13" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 38065 } } peers { permanent_uuid: "534b3bdf0b364f1789738520adfceb2b" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43041 } } peers { permanent_uuid: "eea3c8f431bd4b82be0f155b05811a3f" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45187 } }
I20260812 14:21:44.299494 30796 leader_election.cc:288] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8098227618cb47e6ba40fe20b0a04d13 (127.8.85.3:38065), 534b3bdf0b364f1789738520adfceb2b (127.8.85.2:43041)
I20260812 14:21:44.302349 30784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.299752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:42579 (local address 127.8.85.3:38065)
0812 14:21:44.299824 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:44.299827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:44.299840 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:21:44.299873 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:44.299874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:44.299885 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:44.299979 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:44.300171 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.300649 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.301628 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.301991 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.302073 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.302083 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:44.302231 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 14:21:44.302270 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:21:44.302349 30464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.299667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:38065 (local address 127.8.85.1:42579)
0812 14:21:44.299725 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:44.299738 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:44.299783 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:44.300016 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:44.300018 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:44.300020 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:44.300089 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:44.300091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.300750 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.300751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:44.301531 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:44.301533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.302074 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.302076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:44.302154 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.302158 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:21:44.302226 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 14:21:44.302276 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":3}
I20260812 14:21:44.302438 30787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.299910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:52489 (local address 127.8.85.2:43041)
0812 14:21:44.299977 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:44.299979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:44.300032 (+ 53us) server_negotiation.cc:407] Connection header received
0812 14:21:44.300098 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:44.300099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:44.300108 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:44.300164 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:44.300319 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.300777 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.301714 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.302076 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.302155 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.302164 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:44.302330 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 14:21:44.302365 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:21:44.302465 30799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.299789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:43041 (local address 127.8.85.1:52489)
0812 14:21:44.299964 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:44.299973 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:44.300030 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:44.300168 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:44.300168 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:44.300170 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:44.300240 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:44.300241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.300870 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.300871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:44.301613 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:44.301615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.302159 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.302160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:44.302235 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.302240 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:21:44.302321 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 14:21:44.302359 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":98,"threads_started":1}
I20260812 14:21:44.302736 30714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e551f0f009a491bab0f3db153e93af6" candidate_uuid: "eea3c8f431bd4b82be0f155b05811a3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8098227618cb47e6ba40fe20b0a04d13" is_pre_election: true
I20260812 14:21:44.302858 30559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e551f0f009a491bab0f3db153e93af6" candidate_uuid: "eea3c8f431bd4b82be0f155b05811a3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "534b3bdf0b364f1789738520adfceb2b" is_pre_election: true
I20260812 14:21:44.302913 30714 raft_consensus.cc:2454] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eea3c8f431bd4b82be0f155b05811a3f in term 0.
I20260812 14:21:44.303037 30559 raft_consensus.cc:2454] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eea3c8f431bd4b82be0f155b05811a3f in term 0.
I20260812 14:21:44.303191 30327 leader_election.cc:302] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8098227618cb47e6ba40fe20b0a04d13, eea3c8f431bd4b82be0f155b05811a3f; no voters:
I20260812 14:21:44.303336 30796 raft_consensus.cc:2781] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:21:44.303398 30796 raft_consensus.cc:482] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:21:44.303447 30796 raft_consensus.cc:3037] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:44.304141 30796 raft_consensus.cc:504] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8098227618cb47e6ba40fe20b0a04d13" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 38065 } } peers { permanent_uuid: "534b3bdf0b364f1789738520adfceb2b" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43041 } } peers { permanent_uuid: "eea3c8f431bd4b82be0f155b05811a3f" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45187 } }
I20260812 14:21:44.304270 30796 leader_election.cc:288] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [CANDIDATE]: Term 1 election: Requested vote from peers 8098227618cb47e6ba40fe20b0a04d13 (127.8.85.3:38065), 534b3bdf0b364f1789738520adfceb2b (127.8.85.2:43041)
I20260812 14:21:44.304474 30559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e551f0f009a491bab0f3db153e93af6" candidate_uuid: "eea3c8f431bd4b82be0f155b05811a3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "534b3bdf0b364f1789738520adfceb2b"
I20260812 14:21:44.304507 30714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e551f0f009a491bab0f3db153e93af6" candidate_uuid: "eea3c8f431bd4b82be0f155b05811a3f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8098227618cb47e6ba40fe20b0a04d13"
I20260812 14:21:44.304551 30559 raft_consensus.cc:3037] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:44.304597 30714 raft_consensus.cc:3037] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:44.305279 30559 raft_consensus.cc:2454] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eea3c8f431bd4b82be0f155b05811a3f in term 1.
I20260812 14:21:44.305415 30714 raft_consensus.cc:2454] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eea3c8f431bd4b82be0f155b05811a3f in term 1.
I20260812 14:21:44.305449 30327 leader_election.cc:302] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 534b3bdf0b364f1789738520adfceb2b, eea3c8f431bd4b82be0f155b05811a3f; no voters:
I20260812 14:21:44.305540 30796 raft_consensus.cc:2781] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:44.305721 30796 raft_consensus.cc:686] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [term 1 LEADER]: Becoming Leader. State: Replica: eea3c8f431bd4b82be0f155b05811a3f, State: Running, Role: LEADER
I20260812 14:21:44.305827 30796 consensus_queue.cc:237] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8098227618cb47e6ba40fe20b0a04d13" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 38065 } } peers { permanent_uuid: "534b3bdf0b364f1789738520adfceb2b" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43041 } } peers { permanent_uuid: "eea3c8f431bd4b82be0f155b05811a3f" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45187 } }
I20260812 14:21:44.306761 30236 catalog_manager.cc:5495] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f reported cstate change: term changed from 0 to 1, leader changed from <none> to eea3c8f431bd4b82be0f155b05811a3f (127.8.85.1). New cstate: current_term: 1 leader_uuid: "eea3c8f431bd4b82be0f155b05811a3f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8098227618cb47e6ba40fe20b0a04d13" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 38065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "534b3bdf0b364f1789738520adfceb2b" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eea3c8f431bd4b82be0f155b05811a3f" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 45187 } health_report { overall_health: HEALTHY } } }
I20260812 14:21:44.319885 30785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.317467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42390 (local address 127.8.85.1:45187)
0812 14:21:44.317514 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:44.317516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:44.317534 (+ 18us) server_negotiation.cc:407] Connection header received
0812 14:21:44.317570 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:44.317571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:44.317598 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:44.317677 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:21:44.317803 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.318312 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.319070 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:44.319271 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:44.319350 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:44.319515 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:21:44.319610 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:21:44.319715 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:44.319781 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 14:21:44.319793 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:21:44.321806 30559 raft_consensus.cc:1261] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b [term 1 FOLLOWER]: Refusing update from remote peer eea3c8f431bd4b82be0f155b05811a3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:21:44.321854 30714 raft_consensus.cc:1261] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13 [term 1 FOLLOWER]: Refusing update from remote peer eea3c8f431bd4b82be0f155b05811a3f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:21:44.322268 30800 consensus_queue.cc:1035] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8098227618cb47e6ba40fe20b0a04d13" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 38065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:21:44.322420 30796 consensus_queue.cc:1035] T 0e551f0f009a491bab0f3db153e93af6 P eea3c8f431bd4b82be0f155b05811a3f [LEADER]: Connected to new peer: Peer: permanent_uuid: "534b3bdf0b364f1789738520adfceb2b" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 14:21:44.323490 30619 tablet.cc:2369] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:21:44.325022 30805 mvcc.cc:205] Tried to move back new op lower bound from 7317686289698770944 to 7317686289637527552. Current Snapshot: MvccSnapshot[applied={T|T < 7317686289698770944}]
I20260812 14:21:44.325752 30807 mvcc.cc:205] Tried to move back new op lower bound from 7317686289698770944 to 7317686289637527552. Current Snapshot: MvccSnapshot[applied={T|T < 7317686289698770944}]
I20260812 14:21:44.327780 30236 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:21:44.327886 30236 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:21:44.328688 30236 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d2a51c78461547b182eb731690a219a7: Sending DeleteTablet for 3 replicas of tablet 0e551f0f009a491bab0f3db153e93af6
I20260812 14:21:44.329051 30381 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e551f0f009a491bab0f3db153e93af6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:44 UTC) from {username='slave'} at 127.0.0.1:52094
I20260812 14:21:44.329077 30690 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e551f0f009a491bab0f3db153e93af6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:44 UTC) from {username='slave'} at 127.0.0.1:35434
I20260812 14:21:44.329285 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 30307
I20260812 14:21:44.329339 30534 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e551f0f009a491bab0f3db153e93af6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:44 UTC) from {username='slave'} at 127.0.0.1:49756
I20260812 14:21:44.329840 30818 tablet_replica.cc:330] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13: stopping tablet replica
I20260812 14:21:44.329927 30818 raft_consensus.cc:2229] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:21:44.329998 30818 raft_consensus.cc:2258] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:44.332288 30819 tablet_replica.cc:330] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b: stopping tablet replica
I20260812 14:21:44.332387 30819 raft_consensus.cc:2229] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:21:44.332475 30819 raft_consensus.cc:2258] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:44.332943 30819 ts_tablet_manager.cc:1905] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:21:44.335103 30819 ts_tablet_manager.cc:1918] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:21:44.335201 30819 log.cc:1196] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/0e551f0f009a491bab0f3db153e93af6
I20260812 14:21:44.335603 30819 ts_tablet_manager.cc:1939] T 0e551f0f009a491bab0f3db153e93af6 P 534b3bdf0b364f1789738520adfceb2b: Deleting consensus metadata
I20260812 14:21:44.336037 30218 catalog_manager.cc:4841] TS 534b3bdf0b364f1789738520adfceb2b (127.8.85.2:43041): tablet 0e551f0f009a491bab0f3db153e93af6 (table test-table [id=175dd2c37c6448dd9e67df93f09788e8]) successfully deleted
W20260812 14:21:44.339730 30214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:45187 (error 108)
I20260812 14:21:44.339943 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 30465
I20260812 14:21:44.339944 30818 ts_tablet_manager.cc:1905] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:21:44.341794 30818 ts_tablet_manager.cc:1918] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:21:44.341876 30818 log.cc:1196] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/0e551f0f009a491bab0f3db153e93af6
I20260812 14:21:44.342147 30818 ts_tablet_manager.cc:1939] T 0e551f0f009a491bab0f3db153e93af6 P 8098227618cb47e6ba40fe20b0a04d13: Deleting consensus metadata
I20260812 14:21:44.342278 30786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:44.342120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:45187 (local address 127.0.0.1:42402)
0812 14:21:44.342181 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:44.342226 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:45187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 14:21:44.342494 30214 catalog_manager.cc:4584] TS eea3c8f431bd4b82be0f155b05811a3f (127.8.85.1:45187): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e551f0f009a491bab0f3db153e93af6: Network error: Client connection negotiation failed: client connection to 127.8.85.1:45187: connect: Connection refused (error 111)
I20260812 14:21:44.342609 30218 catalog_manager.cc:4841] TS 8098227618cb47e6ba40fe20b0a04d13 (127.8.85.3:38065): tablet 0e551f0f009a491bab0f3db153e93af6 (table test-table [id=175dd2c37c6448dd9e67df93f09788e8]) successfully deleted
I20260812 14:21:44.347707 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 30621
I20260812 14:21:44.354898 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 30197
2026-08-12T14:21:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (739 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:21:44 dist-test-slave-5k54 krb5kdc[30823](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:21:44 dist-test-slave-5k54 krb5kdc[30823](info): set up 2 sockets
Aug 12 14:21:44 dist-test-slave-5k54 krb5kdc[30823](info): commencing operation
krb5kdc: starting...
W20260812 14:21:45.477650 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:21:45 dist-test-slave-5k54 krb5kdc[30823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:21:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:21:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:21:45.599943 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:38875
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:38875
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:45.682534 30837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:45.682780 30837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:45.682821 30837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:45.683983 30837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:45.684052 30837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:45.684078 30837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:45.684103 30837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:45.684145 30837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:45.685628 30837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:38875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:38875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:45.685909 30837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:45.686131 30837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:45.691859 30848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:45.691903 30844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:45.691908 30843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:45.692078 30837 server_base.cc:1034] running on GCE node
I20260812 14:21:45.692605 30837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:45.692893 30837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:45.694058 30837 hybrid_clock.cc:648] HybridClock initialized: now 1786544505694038 us; error 26 us; skew 500 ppm
Aug 12 14:21:45 dist-test-slave-5k54 krb5kdc[30823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544505, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:45.702361 30837 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:21:45.702752 30837 webserver.cc:460] Webserver started at http://127.8.85.62:35893/ using document root <none> and password file <none>
I20260812 14:21:45.702972 30837 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:45.703020 30837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:45.703114 30837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:45.704229 30837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "aed4be6571ca4c009132fc554736d53f"
format_stamp: "Formatted at 2026-08-12 14:21:45 on dist-test-slave-5k54"
server_key: "ed6fda2f81182a4905f6d86010718101"
server_key_iv: "25f459206a483221de3e11ae5bd12205"
server_key_version: "encryptionkey@0"
I20260812 14:21:45.704558 30837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "aed4be6571ca4c009132fc554736d53f"
format_stamp: "Formatted at 2026-08-12 14:21:45 on dist-test-slave-5k54"
server_key: "ed6fda2f81182a4905f6d86010718101"
server_key_iv: "25f459206a483221de3e11ae5bd12205"
server_key_version: "encryptionkey@0"
I20260812 14:21:45.705840 30837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:21:45.706558 30855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:45.706755 30837 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:21:45.706846 30837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "aed4be6571ca4c009132fc554736d53f"
format_stamp: "Formatted at 2026-08-12 14:21:45 on dist-test-slave-5k54"
server_key: "ed6fda2f81182a4905f6d86010718101"
server_key_iv: "25f459206a483221de3e11ae5bd12205"
server_key_version: "encryptionkey@0"
I20260812 14:21:45.706909 30837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:45.730461 30837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:45.732470 30837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:45.732626 30837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:45.736832 30837 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:38875
I20260812 14:21:45.736869 30919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:38875 every 8 connection(s)
I20260812 14:21:45.737223 30837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:45.738108 30923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:45.740473 30923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f: Bootstrap starting.
I20260812 14:21:45.741060 30923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:45.741355 30923 log.cc:824] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:45.742074 30923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f: No bootstrap required, opened a new log
I20260812 14:21:45.743640 30923 raft_consensus.cc:348] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed4be6571ca4c009132fc554736d53f" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38875 } }
I20260812 14:21:45.743770 30923 raft_consensus.cc:374] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:45.743793 30923 raft_consensus.cc:729] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed4be6571ca4c009132fc554736d53f, State: Initialized, Role: FOLLOWER
I20260812 14:21:45.743963 30923 consensus_queue.cc:260] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed4be6571ca4c009132fc554736d53f" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38875 } }
I20260812 14:21:45.744047 30923 raft_consensus.cc:388] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:45.744089 30923 raft_consensus.cc:482] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:45.744138 30923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:45.744648 30923 raft_consensus.cc:504] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed4be6571ca4c009132fc554736d53f" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38875 } }
I20260812 14:21:45.744779 30923 leader_election.cc:302] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aed4be6571ca4c009132fc554736d53f; no voters:
I20260812 14:21:45.744793 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 30837
I20260812 14:21:45.744954 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:21:45.745003 30923 leader_election.cc:288] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:21:45.745034 30928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:45.745162 30928 raft_consensus.cc:686] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [term 1 LEADER]: Becoming Leader. State: Replica: aed4be6571ca4c009132fc554736d53f, State: Running, Role: LEADER
I20260812 14:21:45.745195 8532 external_mini_cluster.cc:1428] Setting key c745f005ab3200632fdcf24a3a5bab2b
I20260812 14:21:45.745267 30923 sys_catalog.cc:564] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:45.745242 30928 consensus_queue.cc:237] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed4be6571ca4c009132fc554736d53f" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38875 } }
I20260812 14:21:45.745558 30930 sys_catalog.cc:455] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [sys.catalog]: SysCatalogTable state changed. Reason: New leader aed4be6571ca4c009132fc554736d53f. Latest consensus state: current_term: 1 leader_uuid: "aed4be6571ca4c009132fc554736d53f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed4be6571ca4c009132fc554736d53f" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38875 } } }
I20260812 14:21:45.745638 30930 sys_catalog.cc:458] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:45.745862 30929 sys_catalog.cc:455] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aed4be6571ca4c009132fc554736d53f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aed4be6571ca4c009132fc554736d53f" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38875 } } }
I20260812 14:21:45.746025 30929 sys_catalog.cc:458] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:45.746201 30934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:45.746800 30934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:45.748505 30934 catalog_manager.cc:1347] Generated new cluster ID: d03a4f307083435f991c43e3339bbce6
I20260812 14:21:45.748559 30934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:21:45 dist-test-slave-5k54 krb5kdc[30823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:21:45.756861 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:45.745648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45116 (local address 127.8.85.62:38875)
0812 14:21:45.746003 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:45.746013 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:45.746048 (+ 35us) server_negotiation.cc:407] Connection header received
0812 14:21:45.746171 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:45.746178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:45.746238 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:45.747209 (+ 971us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:45.747316 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:45.748631 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:45.749678 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:45.750285 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:45.750394 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:45.753149 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:45.753153 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:45.753153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:45.753166 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:45.755220 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:45.755620 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:45.755624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:45.755624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:45.755751 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:45.755963 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:45.755964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:45.755965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:45.756275 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:45.756435 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:45.756506 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 14:21:45.756640 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":72,"threads_started":1}
I20260812 14:21:45.769297 30934 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:21:45.769995 30934 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:45.781657 30934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f: Generated new TSK 0
I20260812 14:21:45.781852 30934 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.85.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
Entry for principal kudu/127.8.85.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab.
I20260812 14:21:45.811800 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38875
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:45.896740 30954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:45.896934 30954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:45.896958 30954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:45.898015 30954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:45.898048 30954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:45.898074 30954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:21:45.899466 30954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.1.keytab
--principal=kudu/127.8.85.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:45.899667 30954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:45.899832 30954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:45.904641 30961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:45.904716 30962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:45.904824 30964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:45.904852 30954 server_base.cc:1034] running on GCE node
I20260812 14:21:45.905076 30954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:45.905315 30954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:45.906479 30954 hybrid_clock.cc:648] HybridClock initialized: now 1786544505906474 us; error 37 us; skew 500 ppm
Aug 12 14:21:45 dist-test-slave-5k54 krb5kdc[30823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544505, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:45.909098 30954 init.cc:377] Logged in from keytab as kudu/127.8.85.1@KRBTEST.COM (short username kudu)
I20260812 14:21:45.909561 30954 webserver.cc:460] Webserver started at http://127.8.85.1:35885/ using document root <none> and password file <none>
I20260812 14:21:45.909740 30954 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:45.909778 30954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:45.909869 30954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:45.910902 30954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "491c67843e474c3b84d8f5d7451f6027"
format_stamp: "Formatted at 2026-08-12 14:21:45 on dist-test-slave-5k54"
server_key: "7e04142327e4f2658189013142a24312"
server_key_iv: "47df3c0db56cbd1653105e40f432f273"
server_key_version: "encryptionkey@0"
I20260812 14:21:45.911211 30954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "491c67843e474c3b84d8f5d7451f6027"
format_stamp: "Formatted at 2026-08-12 14:21:45 on dist-test-slave-5k54"
server_key: "7e04142327e4f2658189013142a24312"
server_key_iv: "47df3c0db56cbd1653105e40f432f273"
server_key_version: "encryptionkey@0"
I20260812 14:21:45.912563 30954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:21:45.913270 30972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:45.913424 30954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:21:45.913503 30954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "491c67843e474c3b84d8f5d7451f6027"
format_stamp: "Formatted at 2026-08-12 14:21:45 on dist-test-slave-5k54"
server_key: "7e04142327e4f2658189013142a24312"
server_key_iv: "47df3c0db56cbd1653105e40f432f273"
server_key_version: "encryptionkey@0"
I20260812 14:21:45.913559 30954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:45.936690 30954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:45.938458 30954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:45.938584 30954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:45.938768 30954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:45.939013 30954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:45.939042 30954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:45.939064 30954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:45.939075 30954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:45.944782 30954 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:44143
I20260812 14:21:45.944859 31109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:44143 every 8 connection(s)
I20260812 14:21:45.945181 30954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:21:45.947182 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 30954
I20260812 14:21:45.947309 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:21:45.947573 8532 external_mini_cluster.cc:1428] Setting key 542e3e090dced84faba32b1b68886938
Aug 12 14:21:45 dist-test-slave-5k54 krb5kdc[30823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544505, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.1@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:21:45.955106 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:45.945961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:58357 (local address 127.8.85.62:38875)
0812 14:21:45.946031 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:45.946034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:45.946236 (+ 202us) server_negotiation.cc:407] Connection header received
0812 14:21:45.946701 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:45.946702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:45.946713 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:45.946765 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:45.947957 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:45.948587 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:45.949576 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:45.949823 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:45.949896 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:45.952153 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:45.952154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:45.952155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:45.952163 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:45.953659 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:45.954116 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:45.954117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:45.954118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:45.954179 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:45.954451 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:45.954452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:45.954452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:45.954570 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:45.954648 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:45.954923 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 14:21:45.955016 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 14:21:45.955282 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:45.945862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38875 (local address 127.8.85.1:58357)
0812 14:21:45.946062 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:45.946103 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:45.946604 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:45.946823 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:45.946827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:45.947572 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:45.947832 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:45.947840 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:45.948687 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:45.948688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:45.949485 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:45.949487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:45.949951 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:45.949952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:45.950111 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:45.950625 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:21:45.950634 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:45.952057 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:45.953763 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:45.953771 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:45.953773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:45.954024 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:45.954253 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:45.954254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:45.954254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:45.954364 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:45.954639 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:45.954647 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:45.954787 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 14:21:45.954897 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 14:21:45.955051 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
I20260812 14:21:45.955793 31110 heartbeater.cc:347] Connected to a master server at 127.8.85.62:38875
I20260812 14:21:45.955916 31110 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:45.956130 31110 heartbeater.cc:511] Master 127.8.85.62:38875 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.85.2@KRBTEST.COM; defaulting to no policy
I20260812 14:21:45.956696 30875 ts_manager.cc:194] Registered new tserver with Master: 491c67843e474c3b84d8f5d7451f6027 (127.8.85.1:44143)
I20260812 14:21:45.957310 30875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.1@KRBTEST.COM'} at 127.8.85.1:58357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
Entry for principal kudu/127.8.85.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab.
I20260812 14:21:45.974319 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38875
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:46.065325 31115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:46.065523 31115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:46.065546 31115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:46.066613 31115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:46.066649 31115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:46.066679 31115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:21:46.068063 31115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.2.keytab
--principal=kudu/127.8.85.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:46.068238 31115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:46.068416 31115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:46.073496 31121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:46.073627 31115 server_base.cc:1034] running on GCE node
W20260812 14:21:46.073491 31122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:46.073491 31124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:46.074011 31115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:46.074270 31115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:46.075507 31115 hybrid_clock.cc:648] HybridClock initialized: now 1786544506075493 us; error 25 us; skew 500 ppm
Aug 12 14:21:46 dist-test-slave-5k54 krb5kdc[30823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544506, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:46.078190 31115 init.cc:377] Logged in from keytab as kudu/127.8.85.2@KRBTEST.COM (short username kudu)
I20260812 14:21:46.078634 31115 webserver.cc:460] Webserver started at http://127.8.85.2:37381/ using document root <none> and password file <none>
I20260812 14:21:46.078869 31115 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:46.078941 31115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:46.079115 31115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:46.080245 31115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "6fda152e4230481884eb538575911cbf"
format_stamp: "Formatted at 2026-08-12 14:21:46 on dist-test-slave-5k54"
server_key: "ef3a8668dfd750dae619ab694aa9cc65"
server_key_iv: "7af2b8e5615df14d6cd7c29ac144403f"
server_key_version: "encryptionkey@0"
I20260812 14:21:46.080615 31115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "6fda152e4230481884eb538575911cbf"
format_stamp: "Formatted at 2026-08-12 14:21:46 on dist-test-slave-5k54"
server_key: "ef3a8668dfd750dae619ab694aa9cc65"
server_key_iv: "7af2b8e5615df14d6cd7c29ac144403f"
server_key_version: "encryptionkey@0"
I20260812 14:21:46.082065 31115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:21:46.082823 31132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:46.083022 31115 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:21:46.083137 31115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "6fda152e4230481884eb538575911cbf"
format_stamp: "Formatted at 2026-08-12 14:21:46 on dist-test-slave-5k54"
server_key: "ef3a8668dfd750dae619ab694aa9cc65"
server_key_iv: "7af2b8e5615df14d6cd7c29ac144403f"
server_key_version: "encryptionkey@0"
I20260812 14:21:46.083214 31115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:46.109227 31115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:46.111507 31115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:46.111681 31115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:46.111940 31115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:46.112262 31115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:46.112325 31115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:46.112370 31115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:46.112389 31115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:46.118336 31115 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:44653
I20260812 14:21:46.118522 31267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:44653 every 8 connection(s)
I20260812 14:21:46.118767 31115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:21:46.119891 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 31115
I20260812 14:21:46.119988 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:21:46.120208 8532 external_mini_cluster.cc:1428] Setting key c510ac42f5fd7af0cc3381436083e64f
Aug 12 14:21:46 dist-test-slave-5k54 krb5kdc[30823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544506, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.2@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:21:46.129210 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.119566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:58359 (local address 127.8.85.62:38875)
0812 14:21:46.119666 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:46.119668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:46.119814 (+ 146us) server_negotiation.cc:407] Connection header received
0812 14:21:46.120449 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:46.120450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:46.120461 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:46.120523 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:46.121622 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.122366 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.123351 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.123689 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.123769 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.126233 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:46.126235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:46.126235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:46.126243 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:46.127820 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:46.128237 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:46.128238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:46.128239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:46.128305 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:46.128559 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:46.128560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:46.128561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:46.128678 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:46.128758 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:46.129028 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 14:21:46.129120 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:21:46.129417 31270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.119490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38875 (local address 127.8.85.2:58359)
0812 14:21:46.119673 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:46.119716 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:46.120317 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:46.120604 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:46.120608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:46.121316 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:46.121520 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.121526 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.122460 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.122460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.123264 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.123266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.123773 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.123774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.123883 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.124446 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:21:46.124457 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:46.126140 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:46.127922 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:46.127929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:46.127931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:46.128159 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:46.128374 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:46.128375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:46.128375 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:46.128489 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:46.128746 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:46.128753 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:46.128897 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 14:21:46.129006 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 14:21:46.129163 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.8.85.3@KRBTEST.COM; defaulting to no policy
I20260812 14:21:46.129884 31268 heartbeater.cc:347] Connected to a master server at 127.8.85.62:38875
I20260812 14:21:46.129983 31268 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:46.130168 31268 heartbeater.cc:511] Master 127.8.85.62:38875 requested a full tablet report, sending...
I20260812 14:21:46.130558 30875 ts_manager.cc:194] Registered new tserver with Master: 6fda152e4230481884eb538575911cbf (127.8.85.2:44653)
I20260812 14:21:46.130928 30875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.2@KRBTEST.COM'} at 127.8.85.2:58359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
Entry for principal kudu/127.8.85.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab.
I20260812 14:21:46.147716 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38875
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:32963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:46.228578 31273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:46.228816 31273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:46.228855 31273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:46.230006 31273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:46.230073 31273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:46.230118 31273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:21:46.231745 31273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:32963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.3.keytab
--principal=kudu/127.8.85.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:38875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:46.232016 31273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:46.232255 31273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:46.237249 31281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:46.237251 31280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:46.237332 31283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:46.237432 31273 server_base.cc:1034] running on GCE node
I20260812 14:21:46.237665 31273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:46.237978 31273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:46.239145 31273 hybrid_clock.cc:648] HybridClock initialized: now 1786544506239131 us; error 44 us; skew 500 ppm
Aug 12 14:21:46 dist-test-slave-5k54 krb5kdc[30823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544506, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:46.241617 31273 init.cc:377] Logged in from keytab as kudu/127.8.85.3@KRBTEST.COM (short username kudu)
I20260812 14:21:46.242120 31273 webserver.cc:460] Webserver started at http://127.8.85.3:41151/ using document root <none> and password file <none>
I20260812 14:21:46.242353 31273 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:46.242424 31273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:46.242583 31273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:46.243688 31273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "d93e0f982cfb49d2a620157a25dcf123"
format_stamp: "Formatted at 2026-08-12 14:21:46 on dist-test-slave-5k54"
server_key: "9ce3c5fbc5c2efccd01a4d76719fceaf"
server_key_iv: "eb514e7c12cf8732baa672f255bc36dd"
server_key_version: "encryptionkey@0"
I20260812 14:21:46.244060 31273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "d93e0f982cfb49d2a620157a25dcf123"
format_stamp: "Formatted at 2026-08-12 14:21:46 on dist-test-slave-5k54"
server_key: "9ce3c5fbc5c2efccd01a4d76719fceaf"
server_key_iv: "eb514e7c12cf8732baa672f255bc36dd"
server_key_version: "encryptionkey@0"
I20260812 14:21:46.245383 31273 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:21:46.246289 31290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:46.246479 31273 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:21:46.246577 31273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "d93e0f982cfb49d2a620157a25dcf123"
format_stamp: "Formatted at 2026-08-12 14:21:46 on dist-test-slave-5k54"
server_key: "9ce3c5fbc5c2efccd01a4d76719fceaf"
server_key_iv: "eb514e7c12cf8732baa672f255bc36dd"
server_key_version: "encryptionkey@0"
I20260812 14:21:46.246665 31273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:46.256673 31273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:46.258509 31273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:46.258692 31273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:46.258924 31273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:46.259228 31273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:46.259284 31273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:46.259315 31273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:46.259343 31273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:46.265399 31273 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:44297
I20260812 14:21:46.265457 31426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:44297 every 8 connection(s)
I20260812 14:21:46.265751 31273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:21:46 dist-test-slave-5k54 krb5kdc[30823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544506, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.3@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:21:46.272305 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 31273
I20260812 14:21:46.272420 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:21:46.272648 8532 external_mini_cluster.cc:1428] Setting key b6c9efd1efe8c5e6fa30675c5bb5e485
I20260812 14:21:46.275413 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.266458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:45303 (local address 127.8.85.62:38875)
0812 14:21:46.266521 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:46.266524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:46.266740 (+ 216us) server_negotiation.cc:407] Connection header received
0812 14:21:46.267227 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:46.267229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:46.267238 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:46.267289 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:46.268232 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.268719 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.269651 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.269844 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.269909 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.272214 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:46.272216 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:46.272217 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:46.272225 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:46.274088 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:46.274519 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:46.274520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:46.274520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:46.274571 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:46.274832 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:46.274833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:46.274833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:46.274975 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:46.275027 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:46.275286 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 14:21:46.275348 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:21:46.275671 31430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.266440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:38875 (local address 127.8.85.3:45303)
0812 14:21:46.266626 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:46.266661 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:46.267146 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:46.267344 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:46.267347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:46.267965 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:46.268156 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.268161 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.268799 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.268799 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.269580 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.269582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.269951 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.269952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.270049 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.270471 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:21:46.270479 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:46.272064 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:46.274172 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:46.274179 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:46.274181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:46.274449 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:46.274632 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:21:46.274633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:21:46.274633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:21:46.274749 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:21:46.275032 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:46.275038 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:46.275162 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 14:21:46.275258 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 14:21:46.275381 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
I20260812 14:21:46.276132 31427 heartbeater.cc:347] Connected to a master server at 127.8.85.62:38875
I20260812 14:21:46.276226 31427 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:46.276363 31427 heartbeater.cc:511] Master 127.8.85.62:38875 requested a full tablet report, sending...
I20260812 14:21:46.276695 30875 ts_manager.cc:194] Registered new tserver with Master: d93e0f982cfb49d2a620157a25dcf123 (127.8.85.3:44297)
I20260812 14:21:46.277094 30875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.85.3@KRBTEST.COM'} at 127.8.85.3:45303
I20260812 14:21:46.284546 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:21:46.291568 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.285847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45120 (local address 127.8.85.62:38875)
0812 14:21:46.285902 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:46.285904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:46.286033 (+ 129us) server_negotiation.cc:407] Connection header received
0812 14:21:46.286069 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:46.286070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:46.286079 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:46.286130 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:46.286630 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.287193 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.287984 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.288158 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.288210 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.289277 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:46.289278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:46.289278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:46.289284 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:46.290637 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:46.290915 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:46.290916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:46.290916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:46.290964 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:46.291109 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:46.291109 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:46.291110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:46.291227 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:46.291293 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:46.291378 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 14:21:46.291511 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:21:46.293162 30875 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:21:46.294674 30875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:21:46.301154 31440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.296937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:44297 (local address 127.0.0.1:39790)
0812 14:21:46.297040 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:46.297080 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:46.297139 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:46.297847 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:46.297848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:46.297850 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:46.297894 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.297895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.299128 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.299129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.300216 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.300218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.300871 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.300872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.300951 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.300965 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 14:21:46.301032 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 14:21:46.301083 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":3,"thread_start_us":60,"threads_started":1}
I20260812 14:21:46.301162 31443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.297105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39790 (local address 127.8.85.3:44297)
0812 14:21:46.297547 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:46.297551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:46.297561 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:21:46.297596 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:46.297598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:46.297612 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:46.297693 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:46.297986 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.299028 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.300297 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.300767 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.300874 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.300935 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:46.301044 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 14:21:46.301083 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":75,"threads_started":1}
I20260812 14:21:46.301348 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.297379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34870 (local address 127.8.85.1:44143)
0812 14:21:46.297545 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:46.297550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:46.297560 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:21:46.297604 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:46.297605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:46.297617 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:46.297710 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:46.297931 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.298652 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.300101 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.300830 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.300912 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.300964 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:46.301221 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 14:21:46.301268 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
I20260812 14:21:46.301471 31442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.297060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43784 (local address 127.8.85.2:44653)
0812 14:21:46.297289 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:46.297294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:46.297305 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:21:46.297352 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:46.297354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:46.297372 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:46.297475 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:46.297743 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.298875 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.300147 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.300830 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.300902 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.300958 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:46.301313 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 14:21:46.301385 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":75,"threads_started":1}
I20260812 14:21:46.301250 31441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.296946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:44653 (local address 127.0.0.1:43784)
0812 14:21:46.297223 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:46.297236 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:46.297290 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:46.297587 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:46.297590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:46.297592 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:46.297659 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.297661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.299010 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.299011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.300023 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.300026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.300998 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.301000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.301072 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.301078 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 14:21:46.301129 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:21:46.301173 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":40,"thread_start_us":92,"threads_started":1}
I20260812 14:21:46.301559 31444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.297144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:44143 (local address 127.0.0.1:34870)
0812 14:21:46.297383 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:46.297392 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:46.297430 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:46.297799 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:46.297800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:46.297802 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:46.297844 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.297845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.298760 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.298761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.300008 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.300011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.301000 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.301001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.301072 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.301078 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 14:21:46.301129 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:21:46.301173 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":7,"thread_start_us":77,"threads_started":1}
I20260812 14:21:46.302336 31188 tablet_service.cc:1467] Processing CreateTablet for tablet b05bc3798c624103a7978fa8ff3f9c87 (DEFAULT_TABLE table=test-table [id=78a0b28f743d4c2fb397a99ecda34713]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:46.302381 31347 tablet_service.cc:1467] Processing CreateTablet for tablet b05bc3798c624103a7978fa8ff3f9c87 (DEFAULT_TABLE table=test-table [id=78a0b28f743d4c2fb397a99ecda34713]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:46.302691 31347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b05bc3798c624103a7978fa8ff3f9c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:46.302692 31188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b05bc3798c624103a7978fa8ff3f9c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:46.305375 31446 tablet_bootstrap.cc:492] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf: Bootstrap starting.
I20260812 14:21:46.305400 31447 tablet_bootstrap.cc:492] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123: Bootstrap starting.
I20260812 14:21:46.305866 31028 tablet_service.cc:1467] Processing CreateTablet for tablet b05bc3798c624103a7978fa8ff3f9c87 (DEFAULT_TABLE table=test-table [id=78a0b28f743d4c2fb397a99ecda34713]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:46.306063 31447 tablet_bootstrap.cc:654] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:46.306075 31446 tablet_bootstrap.cc:654] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:46.306197 31028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b05bc3798c624103a7978fa8ff3f9c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:46.306602 31447 log.cc:824] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:46.306649 31446 log.cc:824] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:46.307376 31447 tablet_bootstrap.cc:492] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123: No bootstrap required, opened a new log
I20260812 14:21:46.307477 31447 ts_tablet_manager.cc:1397] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:21:46.307651 31446 tablet_bootstrap.cc:492] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf: No bootstrap required, opened a new log
I20260812 14:21:46.307725 31446 ts_tablet_manager.cc:1397] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:21:46.309093 31447 raft_consensus.cc:348] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d93e0f982cfb49d2a620157a25dcf123" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44297 } } peers { permanent_uuid: "6fda152e4230481884eb538575911cbf" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44653 } } peers { permanent_uuid: "491c67843e474c3b84d8f5d7451f6027" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 44143 } }
I20260812 14:21:46.309270 31447 raft_consensus.cc:374] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:46.309317 31447 raft_consensus.cc:729] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d93e0f982cfb49d2a620157a25dcf123, State: Initialized, Role: FOLLOWER
I20260812 14:21:46.309605 31447 consensus_queue.cc:260] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d93e0f982cfb49d2a620157a25dcf123" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44297 } } peers { permanent_uuid: "6fda152e4230481884eb538575911cbf" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44653 } } peers { permanent_uuid: "491c67843e474c3b84d8f5d7451f6027" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 44143 } }
I20260812 14:21:46.309926 31447 ts_tablet_manager.cc:1428] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:21:46.309933 31427 heartbeater.cc:503] Master 127.8.85.62:38875 was elected leader, sending a full tablet report...
I20260812 14:21:46.309878 31446 raft_consensus.cc:348] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d93e0f982cfb49d2a620157a25dcf123" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44297 } } peers { permanent_uuid: "6fda152e4230481884eb538575911cbf" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44653 } } peers { permanent_uuid: "491c67843e474c3b84d8f5d7451f6027" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 44143 } }
I20260812 14:21:46.310061 31446 raft_consensus.cc:374] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:46.310106 31446 raft_consensus.cc:729] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fda152e4230481884eb538575911cbf, State: Initialized, Role: FOLLOWER
I20260812 14:21:46.310215 31446 consensus_queue.cc:260] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d93e0f982cfb49d2a620157a25dcf123" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44297 } } peers { permanent_uuid: "6fda152e4230481884eb538575911cbf" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44653 } } peers { permanent_uuid: "491c67843e474c3b84d8f5d7451f6027" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 44143 } }
I20260812 14:21:46.310509 31446 ts_tablet_manager.cc:1428] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:21:46.310566 31268 heartbeater.cc:503] Master 127.8.85.62:38875 was elected leader, sending a full tablet report...
I20260812 14:21:46.310708 31450 tablet_bootstrap.cc:492] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027: Bootstrap starting.
I20260812 14:21:46.311405 31450 tablet_bootstrap.cc:654] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:46.311919 31450 log.cc:824] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:46.312666 31450 tablet_bootstrap.cc:492] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027: No bootstrap required, opened a new log
I20260812 14:21:46.312745 31450 ts_tablet_manager.cc:1397] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:21:46.314131 31450 raft_consensus.cc:348] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d93e0f982cfb49d2a620157a25dcf123" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44297 } } peers { permanent_uuid: "6fda152e4230481884eb538575911cbf" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44653 } } peers { permanent_uuid: "491c67843e474c3b84d8f5d7451f6027" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 44143 } }
I20260812 14:21:46.314299 31450 raft_consensus.cc:374] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:46.314347 31450 raft_consensus.cc:729] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 491c67843e474c3b84d8f5d7451f6027, State: Initialized, Role: FOLLOWER
I20260812 14:21:46.314463 31450 consensus_queue.cc:260] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d93e0f982cfb49d2a620157a25dcf123" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44297 } } peers { permanent_uuid: "6fda152e4230481884eb538575911cbf" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44653 } } peers { permanent_uuid: "491c67843e474c3b84d8f5d7451f6027" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 44143 } }
I20260812 14:21:46.314720 31450 ts_tablet_manager.cc:1428] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:21:46.314767 31110 heartbeater.cc:503] Master 127.8.85.62:38875 was elected leader, sending a full tablet report...
W20260812 14:21:46.369405 31269 tablet.cc:2369] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:21:46.424314 31454 raft_consensus.cc:482] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:21:46.424467 31454 raft_consensus.cc:504] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d93e0f982cfb49d2a620157a25dcf123" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44297 } } peers { permanent_uuid: "6fda152e4230481884eb538575911cbf" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44653 } } peers { permanent_uuid: "491c67843e474c3b84d8f5d7451f6027" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 44143 } }
I20260812 14:21:46.424949 31454 leader_election.cc:288] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d93e0f982cfb49d2a620157a25dcf123 (127.8.85.3:44297), 6fda152e4230481884eb538575911cbf (127.8.85.2:44653)
I20260812 14:21:46.428054 31455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.425179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:44653 (local address 127.8.85.1:47957)
0812 14:21:46.425344 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:46.425358 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:46.425422 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:46.425567 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:46.425568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:46.425570 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:46.425647 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.425648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.426370 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.426371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.427119 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.427121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.427767 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.427768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.427845 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.427848 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:21:46.427932 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 14:21:46.427972 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":91,"threads_started":1}
I20260812 14:21:46.428043 31442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.425253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:47957 (local address 127.8.85.2:44653)
0812 14:21:46.425300 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:46.425302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:46.425412 (+ 110us) server_negotiation.cc:407] Connection header received
0812 14:21:46.425486 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:46.425487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:46.425500 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:46.425564 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:46.425722 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.426281 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.427216 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.427673 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.427753 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.427764 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:46.427939 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 14:21:46.427978 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:21:46.428146 31443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.425259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:47935 (local address 127.8.85.3:44297)
0812 14:21:46.425330 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:46.425332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:46.425344 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:21:46.425376 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:46.425377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:46.425387 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:46.425463 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:46.425646 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.426148 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.427137 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.427585 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.427673 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.427684 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:46.428042 (+ 358us) server_negotiation.cc:299] Negotiation successful
0812 14:21:46.428083 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 14:21:46.428148 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.425144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:44297 (local address 127.8.85.1:47935)
0812 14:21:46.425204 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:46.425216 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:46.425260 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:46.425470 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:46.425472 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:46.425474 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:46.425555 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.425557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.426234 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.426235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.427043 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.427046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.427672 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.427674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.427758 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.427760 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:21:46.428025 (+ 265us) client_negotiation.cc:240] Negotiation successful
0812 14:21:46.428077 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":48}
I20260812 14:21:46.428550 31370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05bc3798c624103a7978fa8ff3f9c87" candidate_uuid: "491c67843e474c3b84d8f5d7451f6027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d93e0f982cfb49d2a620157a25dcf123" is_pre_election: true
I20260812 14:21:46.428544 31211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05bc3798c624103a7978fa8ff3f9c87" candidate_uuid: "491c67843e474c3b84d8f5d7451f6027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fda152e4230481884eb538575911cbf" is_pre_election: true
I20260812 14:21:46.428683 31370 raft_consensus.cc:2454] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 491c67843e474c3b84d8f5d7451f6027 in term 0.
I20260812 14:21:46.428725 31211 raft_consensus.cc:2454] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 491c67843e474c3b84d8f5d7451f6027 in term 0.
I20260812 14:21:46.428949 30974 leader_election.cc:302] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 491c67843e474c3b84d8f5d7451f6027, d93e0f982cfb49d2a620157a25dcf123; no voters:
I20260812 14:21:46.429080 31454 raft_consensus.cc:2781] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:21:46.429137 31454 raft_consensus.cc:482] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:21:46.429163 31454 raft_consensus.cc:3037] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:46.429898 31454 raft_consensus.cc:504] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d93e0f982cfb49d2a620157a25dcf123" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44297 } } peers { permanent_uuid: "6fda152e4230481884eb538575911cbf" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44653 } } peers { permanent_uuid: "491c67843e474c3b84d8f5d7451f6027" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 44143 } }
I20260812 14:21:46.430053 31454 leader_election.cc:288] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [CANDIDATE]: Term 1 election: Requested vote from peers d93e0f982cfb49d2a620157a25dcf123 (127.8.85.3:44297), 6fda152e4230481884eb538575911cbf (127.8.85.2:44653)
I20260812 14:21:46.430197 31370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05bc3798c624103a7978fa8ff3f9c87" candidate_uuid: "491c67843e474c3b84d8f5d7451f6027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d93e0f982cfb49d2a620157a25dcf123"
I20260812 14:21:46.430281 31370 raft_consensus.cc:3037] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:46.430302 31211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05bc3798c624103a7978fa8ff3f9c87" candidate_uuid: "491c67843e474c3b84d8f5d7451f6027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fda152e4230481884eb538575911cbf"
I20260812 14:21:46.430387 31211 raft_consensus.cc:3037] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:46.431066 31370 raft_consensus.cc:2454] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 491c67843e474c3b84d8f5d7451f6027 in term 1.
I20260812 14:21:46.431236 30974 leader_election.cc:302] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 491c67843e474c3b84d8f5d7451f6027, d93e0f982cfb49d2a620157a25dcf123; no voters:
I20260812 14:21:46.431258 31211 raft_consensus.cc:2454] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 491c67843e474c3b84d8f5d7451f6027 in term 1.
I20260812 14:21:46.431319 31454 raft_consensus.cc:2781] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:46.431511 31454 raft_consensus.cc:686] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [term 1 LEADER]: Becoming Leader. State: Replica: 491c67843e474c3b84d8f5d7451f6027, State: Running, Role: LEADER
I20260812 14:21:46.431622 31454 consensus_queue.cc:237] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d93e0f982cfb49d2a620157a25dcf123" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44297 } } peers { permanent_uuid: "6fda152e4230481884eb538575911cbf" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44653 } } peers { permanent_uuid: "491c67843e474c3b84d8f5d7451f6027" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 44143 } }
I20260812 14:21:46.432512 30875 catalog_manager.cc:5495] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 491c67843e474c3b84d8f5d7451f6027 (127.8.85.1). New cstate: current_term: 1 leader_uuid: "491c67843e474c3b84d8f5d7451f6027" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d93e0f982cfb49d2a620157a25dcf123" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fda152e4230481884eb538575911cbf" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "491c67843e474c3b84d8f5d7451f6027" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 44143 } health_report { overall_health: HEALTHY } } }
I20260812 14:21:46.444603 31445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.442177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34880 (local address 127.8.85.1:44143)
0812 14:21:46.442238 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:46.442241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:46.442250 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:21:46.442317 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:46.442318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:46.442358 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:46.442442 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:21:46.442586 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.443044 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.443870 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.444063 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.444132 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.444294 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:21:46.444391 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:21:46.444490 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:46.444510 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 14:21:46.444528 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 14:21:46.445886 31111 tablet.cc:2369] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:21:46.446451 31370 raft_consensus.cc:1261] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123 [term 1 FOLLOWER]: Refusing update from remote peer 491c67843e474c3b84d8f5d7451f6027: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:21:46.446460 31211 raft_consensus.cc:1261] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf [term 1 FOLLOWER]: Refusing update from remote peer 491c67843e474c3b84d8f5d7451f6027: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:21:46.446795 31454 consensus_queue.cc:1035] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d93e0f982cfb49d2a620157a25dcf123" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 44297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:21:46.446959 31456 consensus_queue.cc:1035] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fda152e4230481884eb538575911cbf" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 44653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:21:46.449168 31459 mvcc.cc:205] Tried to move back new op lower bound from 7317686298401980416 to 7317686298344570880. Current Snapshot: MvccSnapshot[applied={T|T < 7317686298401980416}]
I20260812 14:21:46.449544 31460 mvcc.cc:205] Tried to move back new op lower bound from 7317686298401980416 to 7317686298344570880. Current Snapshot: MvccSnapshot[applied={T|T < 7317686298401980416}]
I20260812 14:21:46.449815 31461 mvcc.cc:205] Tried to move back new op lower bound from 7317686298401980416 to 7317686298344570880. Current Snapshot: MvccSnapshot[applied={T|T < 7317686298401980416}]
I20260812 14:21:46.451771 30874 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:21:46.451871 30874 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:21:46.452629 30874 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aed4be6571ca4c009132fc554736d53f: Sending DeleteTablet for 3 replicas of tablet b05bc3798c624103a7978fa8ff3f9c87
I20260812 14:21:46.452980 31188 tablet_service.cc:1514] Processing DeleteTablet for tablet b05bc3798c624103a7978fa8ff3f9c87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:46 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:43784
I20260812 14:21:46.453045 31347 tablet_service.cc:1514] Processing DeleteTablet for tablet b05bc3798c624103a7978fa8ff3f9c87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:46 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:39790
I20260812 14:21:46.453213 31468 tablet_replica.cc:330] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf: stopping tablet replica
I20260812 14:21:46.453197 31028 tablet_service.cc:1514] Processing DeleteTablet for tablet b05bc3798c624103a7978fa8ff3f9c87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:46 UTC) from {username='kudu', principal='kudu/127.8.85.62@KRBTEST.COM'} at 127.0.0.1:34870
I20260812 14:21:46.453305 31468 raft_consensus.cc:2229] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:21:46.453401 31468 raft_consensus.cc:2258] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:46.453428 31470 tablet_replica.cc:330] T b05bc3798c624103a7978fa8ff3f9c87 P 491c67843e474c3b84d8f5d7451f6027: stopping tablet replica
I20260812 14:21:46.453500 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 30954
I20260812 14:21:46.453824 31468 ts_tablet_manager.cc:1905] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:21:46.457100 31468 ts_tablet_manager.cc:1918] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:21:46.457182 31468 log.cc:1196] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/b05bc3798c624103a7978fa8ff3f9c87
I20260812 14:21:46.457541 31468 ts_tablet_manager.cc:1939] T b05bc3798c624103a7978fa8ff3f9c87 P 6fda152e4230481884eb538575911cbf: Deleting consensus metadata
I20260812 14:21:46.458032 30858 catalog_manager.cc:4841] TS 6fda152e4230481884eb538575911cbf (127.8.85.2:44653): tablet b05bc3798c624103a7978fa8ff3f9c87 (table test-table [id=78a0b28f743d4c2fb397a99ecda34713]) successfully deleted
I20260812 14:21:46.459302 31469 tablet_replica.cc:330] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123: stopping tablet replica
I20260812 14:21:46.459442 31469 raft_consensus.cc:2229] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:21:46.459551 31469 raft_consensus.cc:2258] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:46.459978 31469 ts_tablet_manager.cc:1905] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:21:46.461918 31469 ts_tablet_manager.cc:1918] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:21:46.461998 31469 log.cc:1196] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/b05bc3798c624103a7978fa8ff3f9c87
I20260812 14:21:46.462260 31469 ts_tablet_manager.cc:1939] T b05bc3798c624103a7978fa8ff3f9c87 P d93e0f982cfb49d2a620157a25dcf123: Deleting consensus metadata
I20260812 14:21:46.462836 30857 catalog_manager.cc:4841] TS d93e0f982cfb49d2a620157a25dcf123 (127.8.85.3:44297): tablet b05bc3798c624103a7978fa8ff3f9c87 (table test-table [id=78a0b28f743d4c2fb397a99ecda34713]) successfully deleted
W20260812 14:21:46.465143 30857 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:44143 (error 108)
I20260812 14:21:46.465376 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 31115
I20260812 14:21:46.466780 31441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.465534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:44143 (local address 127.0.0.1:34890)
0812 14:21:46.466697 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:46.466738 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:44143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1143}
W20260812 14:21:46.466922 30857 catalog_manager.cc:4584] TS 491c67843e474c3b84d8f5d7451f6027 (127.8.85.1:44143): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b05bc3798c624103a7978fa8ff3f9c87: Network error: Client connection negotiation failed: client connection to 127.8.85.1:44143: connect: Connection refused (error 111)
I20260812 14:21:46.472119 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 31273
I20260812 14:21:46.478399 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 30837
2026-08-12T14:21:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2125 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T14:21:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:21:46Z Disabled control of system clock
I20260812 14:21:46.508822 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:42419
--webserver_interface=127.8.85.62
--webserver_port=0
--builtin_ntp_servers=127.8.85.20:36189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:42419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:21:46.591140 31477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:46.591344 31477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:46.591372 31477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:46.592541 31477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:46.592583 31477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:46.592597 31477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:46.592608 31477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:46.592665 31477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:46.594010 31477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:42419
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:42419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:46.594257 31477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:46.594558 31477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:46.599239 31483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:46.599234 31487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:46.599411 31477 server_base.cc:1034] running on GCE node
W20260812 14:21:46.599560 31490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:46.599798 31477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:46.600127 31477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:46.601301 31477 hybrid_clock.cc:648] HybridClock initialized: now 1786544506601286 us; error 28 us; skew 500 ppm
I20260812 14:21:46.602522 31477 webserver.cc:460] Webserver started at http://127.8.85.62:35419/ using document root <none> and password file <none>
I20260812 14:21:46.602780 31477 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:46.602851 31477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:46.603008 31477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:46.604144 31477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "a841b7cc03f14940a5575e0248b581a5"
format_stamp: "Formatted at 2026-08-12 14:21:46 on dist-test-slave-5k54"
server_key: "dc60f7d901e10cb6c6a3c70bfb62564e"
server_key_iv: "bbca4dfd7e449280d7822330e10983ce"
server_key_version: "encryptionkey@0"
I20260812 14:21:46.604509 31477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "a841b7cc03f14940a5575e0248b581a5"
format_stamp: "Formatted at 2026-08-12 14:21:46 on dist-test-slave-5k54"
server_key: "dc60f7d901e10cb6c6a3c70bfb62564e"
server_key_iv: "bbca4dfd7e449280d7822330e10983ce"
server_key_version: "encryptionkey@0"
I20260812 14:21:46.605901 31477 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:21:46.606686 31498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:46.606923 31477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:21:46.607046 31477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "a841b7cc03f14940a5575e0248b581a5"
format_stamp: "Formatted at 2026-08-12 14:21:46 on dist-test-slave-5k54"
server_key: "dc60f7d901e10cb6c6a3c70bfb62564e"
server_key_iv: "bbca4dfd7e449280d7822330e10983ce"
server_key_version: "encryptionkey@0"
I20260812 14:21:46.607128 31477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:46.630857 31477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:46.631198 31477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:46.631357 31477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:46.635569 31477 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:42419
I20260812 14:21:46.635640 31557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:42419 every 8 connection(s)
I20260812 14:21:46.636011 31477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:46.636997 31558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:46.639299 31558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5: Bootstrap starting.
I20260812 14:21:46.639926 31558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:46.640245 31558 log.cc:824] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:46.640935 31558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5: No bootstrap required, opened a new log
I20260812 14:21:46.642557 31558 raft_consensus.cc:348] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a841b7cc03f14940a5575e0248b581a5" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42419 } }
I20260812 14:21:46.642697 31558 raft_consensus.cc:374] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:46.642720 31558 raft_consensus.cc:729] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a841b7cc03f14940a5575e0248b581a5, State: Initialized, Role: FOLLOWER
I20260812 14:21:46.642905 31558 consensus_queue.cc:260] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a841b7cc03f14940a5575e0248b581a5" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42419 } }
I20260812 14:21:46.642969 31558 raft_consensus.cc:388] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:46.642992 31558 raft_consensus.cc:482] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:46.643061 31558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:46.643579 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 31477
I20260812 14:21:46.643698 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:21:46.643740 31558 raft_consensus.cc:504] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a841b7cc03f14940a5575e0248b581a5" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42419 } }
I20260812 14:21:46.643855 31558 leader_election.cc:302] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a841b7cc03f14940a5575e0248b581a5; no voters:
I20260812 14:21:46.643926 8532 external_mini_cluster.cc:1428] Setting key f64addf32bcb269cec89ed21d1487c64
I20260812 14:21:46.644026 31558 leader_election.cc:288] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:21:46.644083 31562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:46.644246 31562 raft_consensus.cc:686] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [term 1 LEADER]: Becoming Leader. State: Replica: a841b7cc03f14940a5575e0248b581a5, State: Running, Role: LEADER
I20260812 14:21:46.644294 31558 sys_catalog.cc:564] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:46.644353 31562 consensus_queue.cc:237] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a841b7cc03f14940a5575e0248b581a5" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42419 } }
I20260812 14:21:46.644691 31562 sys_catalog.cc:455] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a841b7cc03f14940a5575e0248b581a5. Latest consensus state: current_term: 1 leader_uuid: "a841b7cc03f14940a5575e0248b581a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a841b7cc03f14940a5575e0248b581a5" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42419 } } }
I20260812 14:21:46.644788 31562 sys_catalog.cc:458] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:46.644821 31564 sys_catalog.cc:455] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a841b7cc03f14940a5575e0248b581a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a841b7cc03f14940a5575e0248b581a5" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 42419 } } }
I20260812 14:21:46.644905 31564 sys_catalog.cc:458] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:46.645756 31570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:46.646277 31570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:46.649931 31570 catalog_manager.cc:1347] Generated new cluster ID: 2e9eafa962354652a4de5af51fc184d5
I20260812 14:21:46.649988 31570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:21:46.650484 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.644594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39486 (local address 127.8.85.62:42419)
0812 14:21:46.645131 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:46.645143 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:46.645176 (+ 33us) server_negotiation.cc:407] Connection header received
0812 14:21:46.645290 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:46.645297 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:46.645379 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:46.645527 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:46.645919 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.647359 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.648237 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.648837 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.648987 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.649682 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:46.649685 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:46.649686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:46.649913 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:46.649958 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:46.649974 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:46.650042 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:46.650169 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 14:21:46.650255 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":61,"threads_started":1}
I20260812 14:21:46.673799 31570 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:21:46.674337 31570 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:46.681751 31570 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5: Generated new TSK 0
I20260812 14:21:46.681964 31570 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:21:46.690552 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.1:0
--local_ip_for_outbound_sockets=127.8.85.1
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:42419
--builtin_ntp_servers=127.8.85.20:36189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:21:46.773893 31584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:46.774106 31584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:46.774128 31584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:46.775278 31584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:46.775319 31584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:46.775344 31584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.1
I20260812 14:21:46.776718 31584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.85.1
--webserver_port=0
--tserver_master_addrs=127.8.85.62:42419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.1
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:46.776969 31584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:46.777268 31584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:46.782192 31591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:46.782248 31584 server_base.cc:1034] running on GCE node
W20260812 14:21:46.782202 31589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:46.782212 31593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:46.782609 31584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:46.782840 31584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:46.784056 31584 hybrid_clock.cc:648] HybridClock initialized: now 1786544506784035 us; error 32 us; skew 500 ppm
I20260812 14:21:46.785377 31584 webserver.cc:460] Webserver started at http://127.8.85.1:41109/ using document root <none> and password file <none>
I20260812 14:21:46.785626 31584 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:46.785701 31584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:46.785856 31584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:46.786962 31584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/data/instance:
uuid: "1b139e05549847b3981f6534406b1d7e"
format_stamp: "Formatted at 2026-08-12 14:21:46 on dist-test-slave-5k54"
server_key: "6f0e37151ac465da9b08a93ec4a96ed2"
server_key_iv: "cc972aa4bc1a3d42c98c8df0ed26164e"
server_key_version: "encryptionkey@0"
I20260812 14:21:46.787335 31584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance:
uuid: "1b139e05549847b3981f6534406b1d7e"
format_stamp: "Formatted at 2026-08-12 14:21:46 on dist-test-slave-5k54"
server_key: "6f0e37151ac465da9b08a93ec4a96ed2"
server_key_iv: "cc972aa4bc1a3d42c98c8df0ed26164e"
server_key_version: "encryptionkey@0"
I20260812 14:21:46.788714 31584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:21:46.789479 31600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:46.789649 31584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:21:46.789755 31584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/wal
uuid: "1b139e05549847b3981f6534406b1d7e"
format_stamp: "Formatted at 2026-08-12 14:21:46 on dist-test-slave-5k54"
server_key: "6f0e37151ac465da9b08a93ec4a96ed2"
server_key_iv: "cc972aa4bc1a3d42c98c8df0ed26164e"
server_key_version: "encryptionkey@0"
I20260812 14:21:46.789831 31584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:46.808346 31584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:46.808645 31584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:46.808795 31584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:46.809028 31584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:46.809360 31584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:46.809418 31584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:46.809450 31584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:46.809482 31584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:46.815387 31584 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.1:35191
I20260812 14:21:46.815485 31732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.1:35191 every 8 connection(s)
I20260812 14:21:46.815819 31584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/data/info.pb
I20260812 14:21:46.816471 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 31584
I20260812 14:21:46.816612 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-0/wal/instance
I20260812 14:21:46.816820 8532 external_mini_cluster.cc:1428] Setting key 45241d3f30ee4ff0b1228314ee8344f8
I20260812 14:21:46.817891 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.2:0
--local_ip_for_outbound_sockets=127.8.85.2
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:42419
--builtin_ntp_servers=127.8.85.20:36189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 14:21:46.821686 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.816596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:54471 (local address 127.8.85.62:42419)
0812 14:21:46.816678 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:46.816681 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:46.816988 (+ 307us) server_negotiation.cc:407] Connection header received
0812 14:21:46.817633 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:46.817634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:46.817648 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:46.817705 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:46.818414 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.818917 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.820329 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.820535 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.820592 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.821428 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:46.821429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:46.821429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:46.821433 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:46.821447 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:46.821451 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:46.821494 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:46.821598 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 14:21:46.821626 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 14:21:46.821972 31735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.816566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:42419 (local address 127.8.85.1:54471)
0812 14:21:46.816856 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:46.816898 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:46.817517 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:46.817781 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:46.817785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:46.817795 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:46.818301 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.818305 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.819016 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.819017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.820221 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.820224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.820621 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.820622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.820767 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.821265 (+ 498us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:21:46.821266 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:46.821304 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:21:46.821333 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:46.821514 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:46.821520 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:46.821524 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:21:46.821593 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 14:21:46.821704 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":115,"threads_started":1}
I20260812 14:21:46.822479 31733 heartbeater.cc:347] Connected to a master server at 127.8.85.62:42419
I20260812 14:21:46.822597 31733 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:46.822808 31733 heartbeater.cc:511] Master 127.8.85.62:42419 requested a full tablet report, sending...
I20260812 14:21:46.823372 31516 ts_manager.cc:194] Registered new tserver with Master: 1b139e05549847b3981f6534406b1d7e (127.8.85.1:35191)
I20260812 14:21:46.824011 31516 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.85.1:54471
W20260812 14:21:46.905539 31737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:46.905738 31737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:46.905758 31737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:46.906901 31737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:46.906940 31737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:46.906965 31737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.2
I20260812 14:21:46.908294 31737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.85.2
--webserver_port=0
--tserver_master_addrs=127.8.85.62:42419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.2
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:46.908528 31737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:46.908820 31737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:46.913916 31747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:46.913923 31744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:46.914017 31745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:46.914108 31737 server_base.cc:1034] running on GCE node
I20260812 14:21:46.914348 31737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:46.914553 31737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:46.915690 31737 hybrid_clock.cc:648] HybridClock initialized: now 1786544506915678 us; error 29 us; skew 500 ppm
I20260812 14:21:46.916945 31737 webserver.cc:460] Webserver started at http://127.8.85.2:43139/ using document root <none> and password file <none>
I20260812 14:21:46.917135 31737 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:46.917174 31737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:46.917265 31737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:46.918262 31737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/data/instance:
uuid: "816d8ac52fad45ecb49c3ec502dedd73"
format_stamp: "Formatted at 2026-08-12 14:21:46 on dist-test-slave-5k54"
server_key: "eceb1ec5172f69b13dfdbb70c61b62d8"
server_key_iv: "6612ac5e25986897dbb536d698625a3d"
server_key_version: "encryptionkey@0"
I20260812 14:21:46.918568 31737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance:
uuid: "816d8ac52fad45ecb49c3ec502dedd73"
format_stamp: "Formatted at 2026-08-12 14:21:46 on dist-test-slave-5k54"
server_key: "eceb1ec5172f69b13dfdbb70c61b62d8"
server_key_iv: "6612ac5e25986897dbb536d698625a3d"
server_key_version: "encryptionkey@0"
I20260812 14:21:46.919826 31737 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:21:46.920475 31753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:46.920641 31737 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:21:46.920749 31737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/wal
uuid: "816d8ac52fad45ecb49c3ec502dedd73"
format_stamp: "Formatted at 2026-08-12 14:21:46 on dist-test-slave-5k54"
server_key: "eceb1ec5172f69b13dfdbb70c61b62d8"
server_key_iv: "6612ac5e25986897dbb536d698625a3d"
server_key_version: "encryptionkey@0"
I20260812 14:21:46.920823 31737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:46.941960 31737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:46.942260 31737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:46.942422 31737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:46.942647 31737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:46.942986 31737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:46.943046 31737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:46.943087 31737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:46.943116 31737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:46.949030 31737 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.2:43379
I20260812 14:21:46.949074 31889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.2:43379 every 8 connection(s)
I20260812 14:21:46.949415 31737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/data/info.pb
I20260812 14:21:46.954285 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.950202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.2:35905 (local address 127.8.85.62:42419)
0812 14:21:46.950248 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:46.950250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:46.950492 (+ 242us) server_negotiation.cc:407] Connection header received
0812 14:21:46.950962 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:46.950964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:46.950973 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:46.951026 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:46.951651 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.952136 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.953060 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.953235 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.953279 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.954094 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:46.954095 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:46.954095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:46.954099 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:46.954109 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:46.954113 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:46.954149 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:46.954209 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 14:21:46.954229 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:21:46.954469 31892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:46.950193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:42419 (local address 127.8.85.2:35905)
0812 14:21:46.950369 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:46.950403 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:46.950858 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:46.951080 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:46.951084 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:46.951091 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:46.951583 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.951588 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.952215 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.952216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.952980 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:46.952982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:46.953301 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:46.953301 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:46.953412 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:46.953975 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:21:46.953975 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:46.954003 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:21:46.954024 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:46.954151 (+ 127us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:46.954154 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:46.954156 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:21:46.954200 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:21:46.954269 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260812 14:21:46.954696 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 31737
I20260812 14:21:46.954813 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/wal/instance
I20260812 14:21:46.955027 8532 external_mini_cluster.cc:1428] Setting key c6c134ef3d05439b17d7915aec3148f2
I20260812 14:21:46.955022 31890 heartbeater.cc:347] Connected to a master server at 127.8.85.62:42419
I20260812 14:21:46.955132 31890 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:46.955332 31890 heartbeater.cc:511] Master 127.8.85.62:42419 requested a full tablet report, sending...
I20260812 14:21:46.955746 31516 ts_manager.cc:194] Registered new tserver with Master: 816d8ac52fad45ecb49c3ec502dedd73 (127.8.85.2:43379)
I20260812 14:21:46.956014 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.85.3:0
--local_ip_for_outbound_sockets=127.8.85.3
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:42419
--builtin_ntp_servers=127.8.85.20:36189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 14:21:46.956185 31516 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.85.2:35905
W20260812 14:21:47.040004 31893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:47.040206 31893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:47.040227 31893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:47.041455 31893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:47.041499 31893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:47.041525 31893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.85.3
I20260812 14:21:47.042826 31893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.85.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.85.3
--webserver_port=0
--tserver_master_addrs=127.8.85.62:42419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.85.3
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:47.043015 31893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:47.043256 31893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:47.048244 31899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:47.048310 31901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:47.048377 31898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:47.048635 31893 server_base.cc:1034] running on GCE node
I20260812 14:21:47.048883 31893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:47.049068 31893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:47.050277 31893 hybrid_clock.cc:648] HybridClock initialized: now 1786544507050263 us; error 28 us; skew 500 ppm
I20260812 14:21:47.051710 31893 webserver.cc:460] Webserver started at http://127.8.85.3:46821/ using document root <none> and password file <none>
I20260812 14:21:47.051985 31893 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:47.052033 31893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:47.052161 31893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:47.053234 31893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/data/instance:
uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6"
format_stamp: "Formatted at 2026-08-12 14:21:47 on dist-test-slave-5k54"
server_key: "a6b9d1b856bb147e47d7d65d0877bebe"
server_key_iv: "24ab77d36a193ec8c9cafa7fb9f7da66"
server_key_version: "encryptionkey@0"
I20260812 14:21:47.053611 31893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance:
uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6"
format_stamp: "Formatted at 2026-08-12 14:21:47 on dist-test-slave-5k54"
server_key: "a6b9d1b856bb147e47d7d65d0877bebe"
server_key_iv: "24ab77d36a193ec8c9cafa7fb9f7da66"
server_key_version: "encryptionkey@0"
I20260812 14:21:47.054994 31893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:21:47.055794 31907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:47.055992 31893 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:47.056095 31893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/wal
uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6"
format_stamp: "Formatted at 2026-08-12 14:21:47 on dist-test-slave-5k54"
server_key: "a6b9d1b856bb147e47d7d65d0877bebe"
server_key_iv: "24ab77d36a193ec8c9cafa7fb9f7da66"
server_key_version: "encryptionkey@0"
I20260812 14:21:47.056169 31893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:47.070688 31893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:47.070992 31893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:47.071139 31893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:47.071377 31893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:21:47.071755 31893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:21:47.071791 31893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:47.071842 31893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:21:47.071869 31893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:47.077696 31893 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.3:33453
I20260812 14:21:47.077747 32038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.3:33453 every 8 connection(s)
I20260812 14:21:47.078081 31893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/data/info.pb
I20260812 14:21:47.081729 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 31893
I20260812 14:21:47.081841 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/wal/instance
I20260812 14:21:47.082021 8532 external_mini_cluster.cc:1428] Setting key 8c93fb927c913e546dfdfc77225d9494
I20260812 14:21:47.083724 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:47.078838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.3:48743 (local address 127.8.85.62:42419)
0812 14:21:47.078904 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:47.078907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:47.079145 (+ 238us) server_negotiation.cc:407] Connection header received
0812 14:21:47.079753 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:47.079755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:47.079769 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:47.079821 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:47.080508 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.081021 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.082318 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.082541 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.082621 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:47.083363 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:47.083364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:47.083365 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:47.083370 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:47.083393 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:47.083397 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:47.083468 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:47.083611 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 14:21:47.083644 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:21:47.083971 32043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:47.078829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:42419 (local address 127.8.85.3:48743)
0812 14:21:47.079023 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:47.079057 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:47.079634 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:47.079898 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:47.079902 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:47.079910 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:21:47.080413 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:47.080417 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.081122 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.081124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:47.082227 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:47.082230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.082627 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.082629 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:47.082763 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:47.083224 (+ 461us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:21:47.083224 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:21:47.083253 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:21:47.083276 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:21:47.083530 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:21:47.083534 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:21:47.083537 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:21:47.083612 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 14:21:47.083714 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260812 14:21:47.084440 32041 heartbeater.cc:347] Connected to a master server at 127.8.85.62:42419
I20260812 14:21:47.084540 32041 heartbeater.cc:464] Registering TS with master...
I20260812 14:21:47.084723 32041 heartbeater.cc:511] Master 127.8.85.62:42419 requested a full tablet report, sending...
I20260812 14:21:47.085119 31516 ts_manager.cc:194] Registered new tserver with Master: 4ba32e4b7f8b4ce8a4b52aac2982f7a6 (127.8.85.3:33453)
I20260812 14:21:47.085489 31516 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.85.3:48743
I20260812 14:21:47.093833 8532 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:21:47.098373 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:47.095353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39498 (local address 127.8.85.62:42419)
0812 14:21:47.095463 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:47.095465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:47.095530 (+ 65us) server_negotiation.cc:407] Connection header received
0812 14:21:47.095598 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:47.095599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:47.095613 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:47.095656 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:47.095847 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.096323 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.097143 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.097343 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.097421 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:47.098119 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:47.098120 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:47.098121 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:47.098125 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:47.098140 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:47.098144 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:47.098195 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:47.098262 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 14:21:47.098287 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 14:21:47.099995 31516 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:21:47.101563 31516 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:21:47.108199 32053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:47.104218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33360 (local address 127.8.85.2:43379)
0812 14:21:47.104527 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:47.104531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:47.104540 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:21:47.104594 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:47.104596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:47.104612 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:47.104686 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:47.104889 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.106012 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.107205 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.107806 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.107897 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:47.107992 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:47.108073 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 14:21:47.108112 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":86,"threads_started":1}
I20260812 14:21:47.108189 32050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:47.103984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:43379 (local address 127.0.0.1:33360)
0812 14:21:47.104247 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:47.104291 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:47.104357 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:47.104704 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:47.104707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:47.104710 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:47.104793 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:47.104795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.106110 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.106111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:47.107120 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:47.107123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.107899 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.107900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:47.107972 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:47.107984 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 14:21:47.108060 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 14:21:47.108103 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":88,"threads_started":1}
I20260812 14:21:47.108204 32049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:47.103985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:33453 (local address 127.0.0.1:41810)
0812 14:21:47.104249 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:47.104289 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:47.104358 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:47.104798 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:47.104799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:47.104801 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:47.104853 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:47.104854 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.105813 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.105814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:47.106990 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:47.106993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.107936 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.107937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:47.107997 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:47.108002 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:21:47.108033 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 14:21:47.108094 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260812 14:21:47.108511 32054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:47.104422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41810 (local address 127.8.85.3:33453)
0812 14:21:47.104606 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:47.104611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:47.104622 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:21:47.104662 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:47.104664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:47.104680 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:47.104941 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:47.105003 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.105727 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.107080 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.107830 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.108245 (+ 415us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:47.108328 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:47.108353 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 14:21:47.108410 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":113,"threads_started":1}
I20260812 14:21:47.108634 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:47.103961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:35191 (local address 127.0.0.1:59860)
0812 14:21:47.104247 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:47.104289 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:47.104358 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:47.105002 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:47.105003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:47.105005 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:47.105042 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:47.105043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.106250 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.106251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:47.107295 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:47.107298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.108326 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.108327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:47.108389 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:47.108392 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:21:47.108522 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 14:21:47.108569 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":57,"threads_started":1}
I20260812 14:21:47.108892 32052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:47.104075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59860 (local address 127.8.85.1:35191)
0812 14:21:47.104842 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:47.104846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:47.104856 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:21:47.104896 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:47.104898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:47.104913 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:47.104991 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:47.105102 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.106153 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.107381 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.108138 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.108203 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:47.108289 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:47.108765 (+ 476us) server_negotiation.cc:299] Negotiation successful
0812 14:21:47.108813 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":90,"threads_started":1}
I20260812 14:21:47.109411 31965 tablet_service.cc:1467] Processing CreateTablet for tablet 01b57331dafd4128888b79ae6a9d3755 (DEFAULT_TABLE table=test-table [id=0202b9d81fb8472b99219b9b62fd6db7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:47.109519 31811 tablet_service.cc:1467] Processing CreateTablet for tablet 01b57331dafd4128888b79ae6a9d3755 (DEFAULT_TABLE table=test-table [id=0202b9d81fb8472b99219b9b62fd6db7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:47.109851 31965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01b57331dafd4128888b79ae6a9d3755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:47.109841 31811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01b57331dafd4128888b79ae6a9d3755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:47.112016 31655 tablet_service.cc:1467] Processing CreateTablet for tablet 01b57331dafd4128888b79ae6a9d3755 (DEFAULT_TABLE table=test-table [id=0202b9d81fb8472b99219b9b62fd6db7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:21:47.112354 31655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01b57331dafd4128888b79ae6a9d3755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:47.113354 32055 tablet_bootstrap.cc:492] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6: Bootstrap starting.
I20260812 14:21:47.114146 32055 tablet_bootstrap.cc:654] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:47.114496 32055 log.cc:824] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:47.115271 32055 tablet_bootstrap.cc:492] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6: No bootstrap required, opened a new log
I20260812 14:21:47.115356 32055 ts_tablet_manager.cc:1397] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:21:47.115713 32058 tablet_bootstrap.cc:492] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e: Bootstrap starting.
I20260812 14:21:47.116371 32058 tablet_bootstrap.cc:654] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:47.116711 32058 log.cc:824] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:47.117043 32056 tablet_bootstrap.cc:492] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73: Bootstrap starting.
I20260812 14:21:47.117544 32058 tablet_bootstrap.cc:492] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e: No bootstrap required, opened a new log
I20260812 14:21:47.117656 32058 ts_tablet_manager.cc:1397] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:21:47.117730 32056 tablet_bootstrap.cc:654] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:47.117997 32055 raft_consensus.cc:348] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b139e05549847b3981f6534406b1d7e" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35191 } } peers { permanent_uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33453 } } peers { permanent_uuid: "816d8ac52fad45ecb49c3ec502dedd73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43379 } }
I20260812 14:21:47.118075 32056 log.cc:824] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:47.118150 32055 raft_consensus.cc:374] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:47.118191 32055 raft_consensus.cc:729] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ba32e4b7f8b4ce8a4b52aac2982f7a6, State: Initialized, Role: FOLLOWER
I20260812 14:21:47.118321 32055 consensus_queue.cc:260] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b139e05549847b3981f6534406b1d7e" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35191 } } peers { permanent_uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33453 } } peers { permanent_uuid: "816d8ac52fad45ecb49c3ec502dedd73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43379 } }
I20260812 14:21:47.118605 32055 ts_tablet_manager.cc:1428] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:21:47.118762 32041 heartbeater.cc:503] Master 127.8.85.62:42419 was elected leader, sending a full tablet report...
I20260812 14:21:47.119189 32056 tablet_bootstrap.cc:492] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73: No bootstrap required, opened a new log
I20260812 14:21:47.119261 32056 ts_tablet_manager.cc:1397] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:21:47.120491 32058 raft_consensus.cc:348] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b139e05549847b3981f6534406b1d7e" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35191 } } peers { permanent_uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33453 } } peers { permanent_uuid: "816d8ac52fad45ecb49c3ec502dedd73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43379 } }
I20260812 14:21:47.120662 32058 raft_consensus.cc:374] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:47.120702 32058 raft_consensus.cc:729] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b139e05549847b3981f6534406b1d7e, State: Initialized, Role: FOLLOWER
I20260812 14:21:47.120831 32058 consensus_queue.cc:260] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b139e05549847b3981f6534406b1d7e" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35191 } } peers { permanent_uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33453 } } peers { permanent_uuid: "816d8ac52fad45ecb49c3ec502dedd73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43379 } }
I20260812 14:21:47.121099 32058 ts_tablet_manager.cc:1428] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:21:47.121145 31733 heartbeater.cc:503] Master 127.8.85.62:42419 was elected leader, sending a full tablet report...
I20260812 14:21:47.121824 32056 raft_consensus.cc:348] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b139e05549847b3981f6534406b1d7e" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35191 } } peers { permanent_uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33453 } } peers { permanent_uuid: "816d8ac52fad45ecb49c3ec502dedd73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43379 } }
I20260812 14:21:47.121997 32056 raft_consensus.cc:374] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:47.122051 32056 raft_consensus.cc:729] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 816d8ac52fad45ecb49c3ec502dedd73, State: Initialized, Role: FOLLOWER
I20260812 14:21:47.122165 32056 consensus_queue.cc:260] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b139e05549847b3981f6534406b1d7e" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35191 } } peers { permanent_uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33453 } } peers { permanent_uuid: "816d8ac52fad45ecb49c3ec502dedd73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43379 } }
I20260812 14:21:47.122430 32056 ts_tablet_manager.cc:1428] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:21:47.122514 31890 heartbeater.cc:503] Master 127.8.85.62:42419 was elected leader, sending a full tablet report...
I20260812 14:21:47.178524 32062 raft_consensus.cc:482] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:21:47.178653 32062 raft_consensus.cc:504] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b139e05549847b3981f6534406b1d7e" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35191 } } peers { permanent_uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33453 } } peers { permanent_uuid: "816d8ac52fad45ecb49c3ec502dedd73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43379 } }
I20260812 14:21:47.178987 32062 leader_election.cc:288] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ba32e4b7f8b4ce8a4b52aac2982f7a6 (127.8.85.3:33453), 816d8ac52fad45ecb49c3ec502dedd73 (127.8.85.2:43379)
I20260812 14:21:47.182844 32054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:47.179274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:33969 (local address 127.8.85.3:33453)
0812 14:21:47.179346 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:47.179348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:47.179620 (+ 272us) server_negotiation.cc:407] Connection header received
0812 14:21:47.179687 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:47.179688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:47.179698 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:47.179754 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:47.179902 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.180612 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.181893 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.182470 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.182540 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:47.182549 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:47.182735 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 14:21:47.182775 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:21:47.182844 32053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:47.179284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.85.1:40017 (local address 127.8.85.2:43379)
0812 14:21:47.179357 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:47.179359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:47.179369 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:21:47.179402 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:47.179403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:47.179414 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:47.179497 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:21:47.179681 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.180462 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.181919 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.182455 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.182540 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:47.182549 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:47.182735 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 14:21:47.182775 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:21:47.182940 31735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:47.179174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.2:43379 (local address 127.8.85.1:40017)
0812 14:21:47.179210 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:47.179222 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:47.179266 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:47.179528 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:47.179529 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:47.179531 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:47.179607 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:47.179610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.180557 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.180558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:47.181795 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:47.181798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.182556 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.182557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:47.182652 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:47.182656 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:21:47.182718 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 14:21:47.182779 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":4}
I20260812 14:21:47.182947 32064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:47.179174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.3:33453 (local address 127.8.85.1:33969)
0812 14:21:47.179531 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:47.179543 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:21:47.179618 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:21:47.179770 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:21:47.179770 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:21:47.179771 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:21:47.179825 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:47.179827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.180672 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.180672 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:47.181805 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:21:47.181807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.182558 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.182559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:21:47.182659 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:47.182661 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:21:47.182718 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 14:21:47.182779 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":176,"thread_start_us":129,"threads_started":1}
I20260812 14:21:47.183334 31988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01b57331dafd4128888b79ae6a9d3755" candidate_uuid: "1b139e05549847b3981f6534406b1d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6" is_pre_election: true
I20260812 14:21:47.183544 31988 raft_consensus.cc:2454] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b139e05549847b3981f6534406b1d7e in term 0.
I20260812 14:21:47.183533 31836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01b57331dafd4128888b79ae6a9d3755" candidate_uuid: "1b139e05549847b3981f6534406b1d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "816d8ac52fad45ecb49c3ec502dedd73" is_pre_election: true
I20260812 14:21:47.183663 31836 raft_consensus.cc:2454] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b139e05549847b3981f6534406b1d7e in term 0.
I20260812 14:21:47.183902 31605 leader_election.cc:302] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b139e05549847b3981f6534406b1d7e, 816d8ac52fad45ecb49c3ec502dedd73; no voters:
I20260812 14:21:47.184094 32062 raft_consensus.cc:2781] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:21:47.184146 32062 raft_consensus.cc:482] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:21:47.184162 32062 raft_consensus.cc:3037] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:47.184911 32062 raft_consensus.cc:504] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b139e05549847b3981f6534406b1d7e" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35191 } } peers { permanent_uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33453 } } peers { permanent_uuid: "816d8ac52fad45ecb49c3ec502dedd73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43379 } }
I20260812 14:21:47.185048 32062 leader_election.cc:288] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [CANDIDATE]: Term 1 election: Requested vote from peers 4ba32e4b7f8b4ce8a4b52aac2982f7a6 (127.8.85.3:33453), 816d8ac52fad45ecb49c3ec502dedd73 (127.8.85.2:43379)
I20260812 14:21:47.185218 31988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01b57331dafd4128888b79ae6a9d3755" candidate_uuid: "1b139e05549847b3981f6534406b1d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6"
I20260812 14:21:47.185242 31836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01b57331dafd4128888b79ae6a9d3755" candidate_uuid: "1b139e05549847b3981f6534406b1d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "816d8ac52fad45ecb49c3ec502dedd73"
I20260812 14:21:47.185318 31988 raft_consensus.cc:3037] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:47.185318 31836 raft_consensus.cc:3037] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:47.186254 31988 raft_consensus.cc:2454] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b139e05549847b3981f6534406b1d7e in term 1.
I20260812 14:21:47.186250 31836 raft_consensus.cc:2454] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b139e05549847b3981f6534406b1d7e in term 1.
I20260812 14:21:47.186424 31605 leader_election.cc:302] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b139e05549847b3981f6534406b1d7e, 816d8ac52fad45ecb49c3ec502dedd73; no voters:
I20260812 14:21:47.186520 32062 raft_consensus.cc:2781] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:47.186663 32062 raft_consensus.cc:686] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [term 1 LEADER]: Becoming Leader. State: Replica: 1b139e05549847b3981f6534406b1d7e, State: Running, Role: LEADER
I20260812 14:21:47.186759 32062 consensus_queue.cc:237] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b139e05549847b3981f6534406b1d7e" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35191 } } peers { permanent_uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33453 } } peers { permanent_uuid: "816d8ac52fad45ecb49c3ec502dedd73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43379 } }
I20260812 14:21:47.187631 31515 catalog_manager.cc:5495] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b139e05549847b3981f6534406b1d7e (127.8.85.1). New cstate: current_term: 1 leader_uuid: "1b139e05549847b3981f6534406b1d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b139e05549847b3981f6534406b1d7e" member_type: VOTER last_known_addr { host: "127.8.85.1" port: 35191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "816d8ac52fad45ecb49c3ec502dedd73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43379 } health_report { overall_health: UNKNOWN } } }
W20260812 14:21:47.199950 31891 tablet.cc:2369] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:21:47.203372 32052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:47.201276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59872 (local address 127.8.85.1:35191)
0812 14:21:47.201328 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:47.201330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:47.201360 (+ 30us) server_negotiation.cc:407] Connection header received
0812 14:21:47.201388 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:47.201389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:47.201407 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:47.201465 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:21:47.201569 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.202013 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.202711 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:47.202918 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:47.202973 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:47.203105 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:21:47.203197 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:21:47.203256 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:47.203288 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 14:21:47.203307 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 14:21:47.205099 31988 raft_consensus.cc:1261] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6 [term 1 FOLLOWER]: Refusing update from remote peer 1b139e05549847b3981f6534406b1d7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:21:47.205111 31836 raft_consensus.cc:1261] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73 [term 1 FOLLOWER]: Refusing update from remote peer 1b139e05549847b3981f6534406b1d7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:21:47.205447 32062 consensus_queue.cc:1035] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ba32e4b7f8b4ce8a4b52aac2982f7a6" member_type: VOTER last_known_addr { host: "127.8.85.3" port: 33453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:21:47.205631 32066 consensus_queue.cc:1035] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "816d8ac52fad45ecb49c3ec502dedd73" member_type: VOTER last_known_addr { host: "127.8.85.2" port: 43379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:21:47.207674 32070 mvcc.cc:205] Tried to move back new op lower bound from 7317686301509632000 to 7317686301437538304. Current Snapshot: MvccSnapshot[applied={T|T < 7317686301509632000}]
I20260812 14:21:47.208180 32072 mvcc.cc:205] Tried to move back new op lower bound from 7317686301509632000 to 7317686301437538304. Current Snapshot: MvccSnapshot[applied={T|T < 7317686301509632000}]
I20260812 14:21:47.208719 32071 mvcc.cc:205] Tried to move back new op lower bound from 7317686301509632000 to 7317686301437538304. Current Snapshot: MvccSnapshot[applied={T|T < 7317686301509632000}]
I20260812 14:21:47.210708 31515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:21:47.210798 31515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:21:47.211580 31515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a841b7cc03f14940a5575e0248b581a5: Sending DeleteTablet for 3 replicas of tablet 01b57331dafd4128888b79ae6a9d3755
I20260812 14:21:47.211894 31655 tablet_service.cc:1514] Processing DeleteTablet for tablet 01b57331dafd4128888b79ae6a9d3755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:47 UTC) from {username='slave'} at 127.0.0.1:59860
I20260812 14:21:47.211920 31811 tablet_service.cc:1514] Processing DeleteTablet for tablet 01b57331dafd4128888b79ae6a9d3755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:47 UTC) from {username='slave'} at 127.0.0.1:33360
I20260812 14:21:47.212155 32082 tablet_replica.cc:330] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73: stopping tablet replica
I20260812 14:21:47.212260 32082 raft_consensus.cc:2229] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:21:47.212265 32083 tablet_replica.cc:330] T 01b57331dafd4128888b79ae6a9d3755 P 1b139e05549847b3981f6534406b1d7e: stopping tablet replica
I20260812 14:21:47.212388 32082 raft_consensus.cc:2258] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:47.212426 31965 tablet_service.cc:1514] Processing DeleteTablet for tablet 01b57331dafd4128888b79ae6a9d3755 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:21:47 UTC) from {username='slave'} at 127.0.0.1:41810
I20260812 14:21:47.212559 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 31584
I20260812 14:21:47.216094 32082 ts_tablet_manager.cc:1905] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:21:47.218374 32082 ts_tablet_manager.cc:1918] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:21:47.218482 32082 log.cc:1196] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-1/wal/wals/01b57331dafd4128888b79ae6a9d3755
I20260812 14:21:47.218815 32082 ts_tablet_manager.cc:1939] T 01b57331dafd4128888b79ae6a9d3755 P 816d8ac52fad45ecb49c3ec502dedd73: Deleting consensus metadata
I20260812 14:21:47.219337 31502 catalog_manager.cc:4841] TS 816d8ac52fad45ecb49c3ec502dedd73 (127.8.85.2:43379): tablet 01b57331dafd4128888b79ae6a9d3755 (table test-table [id=0202b9d81fb8472b99219b9b62fd6db7]) successfully deleted
I20260812 14:21:47.220523 32085 tablet_replica.cc:330] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6: stopping tablet replica
I20260812 14:21:47.220664 32085 raft_consensus.cc:2229] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:21:47.220821 32085 raft_consensus.cc:2258] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:21:47.224422 32085 ts_tablet_manager.cc:1905] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:21:47.226456 32085 ts_tablet_manager.cc:1918] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:21:47.226549 32085 log.cc:1196] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6: Deleting WAL directory at /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786544263511571-8532-0/minicluster-data/ts-2/wal/wals/01b57331dafd4128888b79ae6a9d3755
I20260812 14:21:47.226828 32085 ts_tablet_manager.cc:1939] T 01b57331dafd4128888b79ae6a9d3755 P 4ba32e4b7f8b4ce8a4b52aac2982f7a6: Deleting consensus metadata
I20260812 14:21:47.227535 31499 catalog_manager.cc:4841] TS 4ba32e4b7f8b4ce8a4b52aac2982f7a6 (127.8.85.3:33453): tablet 01b57331dafd4128888b79ae6a9d3755 (table test-table [id=0202b9d81fb8472b99219b9b62fd6db7]) successfully deleted
W20260812 14:21:47.229065 31501 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.85.1:35191 (error 108)
I20260812 14:21:47.229599 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:47.229451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.1:35191 (local address 127.0.0.1:59876)
0812 14:21:47.229514 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 14:21:47.229559 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.1:35191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 14:21:47.229701 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 31737
W20260812 14:21:47.229781 31501 catalog_manager.cc:4584] TS 1b139e05549847b3981f6534406b1d7e (127.8.85.1:35191): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 01b57331dafd4128888b79ae6a9d3755: Network error: Client connection negotiation failed: client connection to 127.8.85.1:35191: connect: Connection refused (error 111)
I20260812 14:21:47.237421 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 31893
I20260812 14:21:47.244066 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 31477
2026-08-12T14:21:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (765 ms)
[----------] 6 tests from EncryptionPolicyTest (8715 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:21:47 dist-test-slave-5k54 krb5kdc[32091](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:21:47 dist-test-slave-5k54 krb5kdc[32091](info): set up 2 sockets
Aug 12 14:21:47 dist-test-slave-5k54 krb5kdc[32091](info): commencing operation
krb5kdc: starting...
W20260812 14:21:48.366485 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:21:48 dist-test-slave-5k54 krb5kdc[32091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:21:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:21:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:21:48.490681 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43721
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:48.574189 32111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:48.574404 32111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:48.574426 32111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:48.575662 32111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:48.575709 32111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:48.575724 32111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:48.575736 32111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:48.575749 32111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:48.577260 32111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43721
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:48.577510 32111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:48.577761 32111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:48.582780 32119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:48.582789 32120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:48.582790 32124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:48.583184 32111 server_base.cc:1034] running on GCE node
I20260812 14:21:48.583492 32111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:48.583801 32111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:48.584995 32111 hybrid_clock.cc:648] HybridClock initialized: now 1786544508584983 us; error 29 us; skew 500 ppm
Aug 12 14:21:48 dist-test-slave-5k54 krb5kdc[32091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544508, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:48.587555 32111 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:21:48.588057 32111 webserver.cc:460] Webserver started at http://127.0.0.1:46345/ using document root <none> and password file <none>
I20260812 14:21:48.588299 32111 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:48.588371 32111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:48.588510 32111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:48.589627 32111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "76eed8db5ede4afababbb04f27863e8d"
format_stamp: "Formatted at 2026-08-12 14:21:48 on dist-test-slave-5k54"
server_key: "006727dda90975048bd73824699fbc62"
server_key_iv: "bbcf55c96202ab4fc9c307ca2edf6dda"
server_key_version: "encryptionkey@0"
I20260812 14:21:48.590008 32111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "76eed8db5ede4afababbb04f27863e8d"
format_stamp: "Formatted at 2026-08-12 14:21:48 on dist-test-slave-5k54"
server_key: "006727dda90975048bd73824699fbc62"
server_key_iv: "bbcf55c96202ab4fc9c307ca2edf6dda"
server_key_version: "encryptionkey@0"
I20260812 14:21:48.591369 32111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:21:48.592213 32132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:48.592396 32111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:21:48.592512 32111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "76eed8db5ede4afababbb04f27863e8d"
format_stamp: "Formatted at 2026-08-12 14:21:48 on dist-test-slave-5k54"
server_key: "006727dda90975048bd73824699fbc62"
server_key_iv: "bbcf55c96202ab4fc9c307ca2edf6dda"
server_key_version: "encryptionkey@0"
I20260812 14:21:48.592588 32111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:48.621721 32111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:48.623683 32111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:48.623844 32111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:48.628109 32111 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43721
I20260812 14:21:48.628145 32197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43721 every 8 connection(s)
I20260812 14:21:48.628527 32111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:48.629609 32198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:48.631973 32198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d: Bootstrap starting.
I20260812 14:21:48.632565 32198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:48.632855 32198 log.cc:824] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:48.633579 32198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d: No bootstrap required, opened a new log
I20260812 14:21:48.635380 32198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76eed8db5ede4afababbb04f27863e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43721 } }
I20260812 14:21:48.635598 32198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:48.635677 32198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76eed8db5ede4afababbb04f27863e8d, State: Initialized, Role: FOLLOWER
I20260812 14:21:48.635874 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 32111
I20260812 14:21:48.635874 32198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76eed8db5ede4afababbb04f27863e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43721 } }
I20260812 14:21:48.635980 32198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:48.636025 32198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:48.636009 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:21:48.636078 32198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:48.636231 8532 external_mini_cluster.cc:1428] Setting key 2a4d0df783235f2ea1fd120e43b59648
I20260812 14:21:48.636693 32198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76eed8db5ede4afababbb04f27863e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43721 } }
I20260812 14:21:48.636811 32198 leader_election.cc:302] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76eed8db5ede4afababbb04f27863e8d; no voters:
I20260812 14:21:48.636977 32198 leader_election.cc:288] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:21:48.637064 32203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:48.637200 32203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [term 1 LEADER]: Becoming Leader. State: Replica: 76eed8db5ede4afababbb04f27863e8d, State: Running, Role: LEADER
I20260812 14:21:48.637250 32198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:48.637295 32203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76eed8db5ede4afababbb04f27863e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43721 } }
I20260812 14:21:48.637792 32206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76eed8db5ede4afababbb04f27863e8d. Latest consensus state: current_term: 1 leader_uuid: "76eed8db5ede4afababbb04f27863e8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76eed8db5ede4afababbb04f27863e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43721 } } }
I20260812 14:21:48.637883 32206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:48.638108 32213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:48.638664 32213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:48.639201 32205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76eed8db5ede4afababbb04f27863e8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76eed8db5ede4afababbb04f27863e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43721 } } }
I20260812 14:21:48.639303 32205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d [sys.catalog]: This master's current role is: LEADER
Aug 12 14:21:48 dist-test-slave-5k54 krb5kdc[32091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 14:21:48.643823 32213 catalog_manager.cc:1347] Generated new cluster ID: eb9526530e3e468fb50372efa88887a1
I20260812 14:21:48.643894 32213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:21:48.647603 32202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:48.636762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57509 (local address 127.0.0.1:43721)
0812 14:21:48.636980 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:48.637044 (+ 64us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:48.637080 (+ 36us) server_negotiation.cc:407] Connection header received
0812 14:21:48.637205 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:48.637212 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:48.637270 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:48.637418 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:48.638285 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:48.639760 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:48.640579 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:48.641186 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:48.641302 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:48.643803 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:48.643806 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:48.643807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:48.643817 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:48.645839 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:48.646312 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:48.646315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:48.646316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:48.646429 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:48.646680 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:48.646681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:48.646681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:48.646963 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:48.647137 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:48.647271 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 14:21:48.647374 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":90,"threads_started":1}
I20260812 14:21:48.648504 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:21:48.657232 32202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:48.649946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55769 (local address 127.0.0.1:43721)
0812 14:21:48.649979 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:48.649980 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:48.650096 (+ 116us) server_negotiation.cc:407] Connection header received
0812 14:21:48.650152 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:48.650153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:48.650162 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:48.650217 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:48.650814 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:48.651664 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:48.652685 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:48.653008 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:48.653072 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:48.654408 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:48.654409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:48.654410 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:48.654416 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:48.655948 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:48.656392 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:48.656394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:48.656394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:48.656458 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:48.656703 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:48.656704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:48.656704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:48.656819 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:48.656910 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:48.657085 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 14:21:48.657149 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 14:21:48.660514 32213 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:21:48.661017 32213 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:48.668433 32213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76eed8db5ede4afababbb04f27863e8d: Generated new TSK 0
I20260812 14:21:48.668606 32213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:21:48.672022 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 32111
W20260812 14:21:48.679517 32088 connection.cc:535] client connection to 127.0.0.1:43721 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43721): Connection reset by peer (error 104)
2026-08-12T14:21:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:21:48 dist-test-slave-5k54 krb5kdc[32237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:21:48 dist-test-slave-5k54 krb5kdc[32237](info): set up 2 sockets
Aug 12 14:21:48 dist-test-slave-5k54 krb5kdc[32237](info): commencing operation
krb5kdc: starting...
W20260812 14:21:49.793594 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:21:49 dist-test-slave-5k54 krb5kdc[32237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:21:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:21:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:21:49.930431 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42237
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:50.013942 32256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:50.014201 32256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:50.014240 32256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:50.015452 32256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:50.015527 32256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:50.015552 32256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:50.015579 32256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:50.015609 32256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:50.017086 32256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42237
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:50.017377 32256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:50.017614 32256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:50.022542 32261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:50.022547 32262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:50.022559 32264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:50.022822 32256 server_base.cc:1034] running on GCE node
I20260812 14:21:50.023157 32256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:50.023522 32256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:50.024695 32256 hybrid_clock.cc:648] HybridClock initialized: now 1786544510024678 us; error 27 us; skew 500 ppm
Aug 12 14:21:50 dist-test-slave-5k54 krb5kdc[32237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544510, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:50.027297 32256 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:21:50.027769 32256 webserver.cc:460] Webserver started at http://127.0.0.1:33505/ using document root <none> and password file <none>
I20260812 14:21:50.028015 32256 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:50.028062 32256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:50.028205 32256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:50.029287 32256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "00050337639a4fe4b89fbc67de62e073"
format_stamp: "Formatted at 2026-08-12 14:21:50 on dist-test-slave-5k54"
server_key: "597368fcc0094fbdb73d6ce19930de68"
server_key_iv: "e5b9a8dd6b0c925f98f320aa07358cfb"
server_key_version: "encryptionkey@0"
I20260812 14:21:50.029686 32256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "00050337639a4fe4b89fbc67de62e073"
format_stamp: "Formatted at 2026-08-12 14:21:50 on dist-test-slave-5k54"
server_key: "597368fcc0094fbdb73d6ce19930de68"
server_key_iv: "e5b9a8dd6b0c925f98f320aa07358cfb"
server_key_version: "encryptionkey@0"
I20260812 14:21:50.031242 32256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:21:50.032161 32272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:50.032352 32256 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:21:50.032459 32256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "00050337639a4fe4b89fbc67de62e073"
format_stamp: "Formatted at 2026-08-12 14:21:50 on dist-test-slave-5k54"
server_key: "597368fcc0094fbdb73d6ce19930de68"
server_key_iv: "e5b9a8dd6b0c925f98f320aa07358cfb"
server_key_version: "encryptionkey@0"
I20260812 14:21:50.032555 32256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:50.048449 32256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:50.050344 32256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:50.050514 32256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:50.054736 32256 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42237
I20260812 14:21:50.054812 32330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42237 every 8 connection(s)
I20260812 14:21:50.055156 32256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:50.055984 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 32256
I20260812 14:21:50.056105 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:21:50.056203 32331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:50.056348 8532 external_mini_cluster.cc:1428] Setting key 735942d6ea2365979d1746cbb31af442
I20260812 14:21:50.059353 32331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073: Bootstrap starting.
I20260812 14:21:50.060087 32331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:50.060387 32331 log.cc:824] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:50.061151 32331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073: No bootstrap required, opened a new log
I20260812 14:21:50.061954 32334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:50.056887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60105 (local address 127.0.0.1:42237)
0812 14:21:50.057086 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:50.057095 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:50.057119 (+ 24us) server_negotiation.cc:407] Connection header received
0812 14:21:50.057673 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:50.057680 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:50.057727 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:50.057839 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:50.058088 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:50.058936 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:50.060031 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:50.060509 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:50.060598 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:50.061354 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:50.061358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:50.061358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:50.061524 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:50.061552 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:50.061563 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:50.061656 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:50.061738 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 14:21:50.061767 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
I20260812 14:21:50.063004 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:21:50.063491 32331 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00050337639a4fe4b89fbc67de62e073" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42237 } }
I20260812 14:21:50.063657 32331 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:50.063704 32331 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00050337639a4fe4b89fbc67de62e073, State: Initialized, Role: FOLLOWER
I20260812 14:21:50.063827 32331 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00050337639a4fe4b89fbc67de62e073" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42237 } }
I20260812 14:21:50.063926 32331 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:50.063969 32331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:50.064016 32331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:50.064630 32331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00050337639a4fe4b89fbc67de62e073" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42237 } }
I20260812 14:21:50.064744 32331 leader_election.cc:302] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00050337639a4fe4b89fbc67de62e073; no voters:
I20260812 14:21:50.064955 32331 leader_election.cc:288] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:21:50.064983 32342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:50.065099 32342 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [term 1 LEADER]: Becoming Leader. State: Replica: 00050337639a4fe4b89fbc67de62e073, State: Running, Role: LEADER
I20260812 14:21:50.065187 32342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00050337639a4fe4b89fbc67de62e073" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42237 } }
I20260812 14:21:50.065213 32331 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:50.065613 32343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00050337639a4fe4b89fbc67de62e073" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00050337639a4fe4b89fbc67de62e073" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42237 } } }
I20260812 14:21:50.065711 32343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:50.065927 32350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:50.065963 32344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00050337639a4fe4b89fbc67de62e073. Latest consensus state: current_term: 1 leader_uuid: "00050337639a4fe4b89fbc67de62e073" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00050337639a4fe4b89fbc67de62e073" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42237 } } }
I20260812 14:21:50.066105 32344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:50.066465 32350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:50.068013 32334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:50.064489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60727 (local address 127.0.0.1:42237)
0812 14:21:50.064534 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:50.064536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:50.064581 (+ 45us) server_negotiation.cc:407] Connection header received
0812 14:21:50.064651 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:50.064652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:50.064661 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:50.064707 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:50.064927 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:50.065626 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:50.066736 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:50.066951 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:50.067031 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:50.067766 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:50.067768 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:50.067768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:50.067772 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:50.067785 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:50.067789 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:50.067844 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:50.067904 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 14:21:50.067938 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:21:50.068166 32350 catalog_manager.cc:1347] Generated new cluster ID: 1c85d98bc9cc4761bb09d58acc3c826b
I20260812 14:21:50.068220 32350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:21:50.081012 32350 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:21:50.081718 32350 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:50.087767 32350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00050337639a4fe4b89fbc67de62e073: Generated new TSK 0
I20260812 14:21:50.087963 32350 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:21:50.097855 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 32256
W20260812 14:21:50.104583 32235 connection.cc:535] client connection to 127.0.0.1:42237 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42237): Connection reset by peer (error 104)
2026-08-12T14:21:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:21:50 dist-test-slave-5k54 krb5kdc[32364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:21:50 dist-test-slave-5k54 krb5kdc[32364](info): set up 2 sockets
Aug 12 14:21:50 dist-test-slave-5k54 krb5kdc[32364](info): commencing operation
krb5kdc: starting...
W20260812 14:21:51.212520 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:21:51 dist-test-slave-5k54 krb5kdc[32364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:21:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:21:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:21:51.338116 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:51.421284 32380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:51.421535 32380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:51.421576 32380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:51.422737 32380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:51.422806 32380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:51.422832 32380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:51.422860 32380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:51.422883 32380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:51.425014 32380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37323
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:51.425294 32380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:51.425554 32380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:51.430462 32389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:51.430495 32387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:51.430459 32386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:51.430701 32380 server_base.cc:1034] running on GCE node
I20260812 14:21:51.431015 32380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:51.431310 32380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:51.432484 32380 hybrid_clock.cc:648] HybridClock initialized: now 1786544511432469 us; error 46 us; skew 500 ppm
Aug 12 14:21:51 dist-test-slave-5k54 krb5kdc[32364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544511, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:51.440146 32380 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:21:51.440578 32380 webserver.cc:460] Webserver started at http://127.0.0.1:36373/ using document root <none> and password file <none>
I20260812 14:21:51.440819 32380 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:51.440862 32380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:51.440953 32380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:51.441977 32380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "5ffe24b9c1c4421d9489add253dc3cb9"
format_stamp: "Formatted at 2026-08-12 14:21:51 on dist-test-slave-5k54"
server_key: "d3aaa71df28c03b9de27460cb59f9e87"
server_key_iv: "b8a6fca279cc15e66703fd8c445a8df8"
server_key_version: "encryptionkey@0"
I20260812 14:21:51.442276 32380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "5ffe24b9c1c4421d9489add253dc3cb9"
format_stamp: "Formatted at 2026-08-12 14:21:51 on dist-test-slave-5k54"
server_key: "d3aaa71df28c03b9de27460cb59f9e87"
server_key_iv: "b8a6fca279cc15e66703fd8c445a8df8"
server_key_version: "encryptionkey@0"
I20260812 14:21:51.443567 32380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:21:51.444334 32397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:51.444514 32380 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:21:51.444600 32380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "5ffe24b9c1c4421d9489add253dc3cb9"
format_stamp: "Formatted at 2026-08-12 14:21:51 on dist-test-slave-5k54"
server_key: "d3aaa71df28c03b9de27460cb59f9e87"
server_key_iv: "b8a6fca279cc15e66703fd8c445a8df8"
server_key_version: "encryptionkey@0"
I20260812 14:21:51.444657 32380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:51.447907 32380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:51.449738 32380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:51.449846 32380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:51.453910 32380 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37323
I20260812 14:21:51.453935 32463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37323 every 8 connection(s)
I20260812 14:21:51.454286 32380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:51.455324 32464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:51.457616 32464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9: Bootstrap starting.
I20260812 14:21:51.458191 32464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:51.458490 32464 log.cc:824] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:51.459185 32464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9: No bootstrap required, opened a new log
I20260812 14:21:51.460872 32464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ffe24b9c1c4421d9489add253dc3cb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37323 } }
I20260812 14:21:51.461036 32464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:51.461087 32464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ffe24b9c1c4421d9489add253dc3cb9, State: Initialized, Role: FOLLOWER
I20260812 14:21:51.461241 32464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ffe24b9c1c4421d9489add253dc3cb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37323 } }
I20260812 14:21:51.461330 32464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:51.461372 32464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:51.461421 32464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:51.461967 32464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ffe24b9c1c4421d9489add253dc3cb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37323 } }
I20260812 14:21:51.462090 32464 leader_election.cc:302] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ffe24b9c1c4421d9489add253dc3cb9; no voters:
I20260812 14:21:51.462275 32464 leader_election.cc:288] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:21:51.462317 32467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:51.462532 32467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [term 1 LEADER]: Becoming Leader. State: Replica: 5ffe24b9c1c4421d9489add253dc3cb9, State: Running, Role: LEADER
I20260812 14:21:51.462569 32464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:51.462632 32467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ffe24b9c1c4421d9489add253dc3cb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37323 } }
I20260812 14:21:51.462991 32468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ffe24b9c1c4421d9489add253dc3cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ffe24b9c1c4421d9489add253dc3cb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37323 } } }
I20260812 14:21:51.462994 32469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ffe24b9c1c4421d9489add253dc3cb9. Latest consensus state: current_term: 1 leader_uuid: "5ffe24b9c1c4421d9489add253dc3cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ffe24b9c1c4421d9489add253dc3cb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37323 } } }
I20260812 14:21:51.463133 32468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:51.463151 32469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:51.463305 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 32380
I20260812 14:21:51.463408 32471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:51.463460 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:21:51.463677 8532 external_mini_cluster.cc:1428] Setting key f9808d37d8a62993f40d6c269fb5b4ad
I20260812 14:21:51.463964 32471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:51.466337 32471 catalog_manager.cc:1347] Generated new cluster ID: 64657e07bc2d4f0b8c1a23a449b0476e
I20260812 14:21:51.466391 32471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:21:51.466946 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:51.464396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37795 (local address 127.0.0.1:37323)
0812 14:21:51.465270 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:51.465280 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:51.465314 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:21:51.465503 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:51.465511 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:51.465546 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:51.465666 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:51.466489 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:51.466493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:51.466493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:51.466498 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:51.466538 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:51.466552 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:51.466589 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:51.466729 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 14:21:51.466770 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":817,"thread_start_us":65,"threads_started":1}
I20260812 14:21:51.468035 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:21:51.470504 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:51.469458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50523 (local address 127.0.0.1:37323)
0812 14:21:51.469488 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:51.469489 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:51.469636 (+ 147us) server_negotiation.cc:407] Connection header received
0812 14:21:51.469676 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:51.469677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:51.469682 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:51.469733 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:51.470293 (+ 560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:51.470294 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:51.470294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:51.470298 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:51.470313 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:51.470317 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:51.470378 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:51.470435 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 14:21:51.470447 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 14:21:51.477751 32471 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:21:51.477911 32471 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:51.483937 32471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ffe24b9c1c4421d9489add253dc3cb9: Generated new TSK 0
I20260812 14:21:51.484130 32471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:21:51.488749 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 32380
2026-08-12T14:21:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1391 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:21:51 dist-test-slave-5k54 krb5kdc[32501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:21:51 dist-test-slave-5k54 krb5kdc[32501](info): set up 2 sockets
Aug 12 14:21:51 dist-test-slave-5k54 krb5kdc[32501](info): commencing operation
krb5kdc: starting...
W20260812 14:21:52.613672 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:21:52 dist-test-slave-5k54 krb5kdc[32501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:21:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:21:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:21:52.743261 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42951
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:52.827122 32517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:52.827389 32517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:52.827435 32517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:52.828580 32517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:52.828655 32517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:52.828677 32517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:52.828698 32517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:52.828729 32517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:52.830212 32517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:52.830504 32517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:52.830742 32517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:52.835611 32524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:52.835675 32527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:52.835657 32525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:52.835927 32517 server_base.cc:1034] running on GCE node
I20260812 14:21:52.836148 32517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:52.836388 32517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:52.837555 32517 hybrid_clock.cc:648] HybridClock initialized: now 1786544512837536 us; error 35 us; skew 500 ppm
Aug 12 14:21:52 dist-test-slave-5k54 krb5kdc[32501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544512, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:52.840152 32517 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:21:52.840651 32517 webserver.cc:460] Webserver started at http://127.0.0.1:42431/ using document root <none> and password file <none>
I20260812 14:21:52.840889 32517 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:52.840960 32517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:52.841115 32517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:52.842228 32517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "b251345ce1b94dfcba0f1b504952a4fe"
format_stamp: "Formatted at 2026-08-12 14:21:52 on dist-test-slave-5k54"
server_key: "4887416b8c2717ab29d291942f8f8741"
server_key_iv: "e1a987129addb022a330f7f89b672eca"
server_key_version: "encryptionkey@0"
I20260812 14:21:52.842614 32517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "b251345ce1b94dfcba0f1b504952a4fe"
format_stamp: "Formatted at 2026-08-12 14:21:52 on dist-test-slave-5k54"
server_key: "4887416b8c2717ab29d291942f8f8741"
server_key_iv: "e1a987129addb022a330f7f89b672eca"
server_key_version: "encryptionkey@0"
I20260812 14:21:52.844125 32517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:21:52.844885 32538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:52.845064 32517 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:21:52.845172 32517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "b251345ce1b94dfcba0f1b504952a4fe"
format_stamp: "Formatted at 2026-08-12 14:21:52 on dist-test-slave-5k54"
server_key: "4887416b8c2717ab29d291942f8f8741"
server_key_iv: "e1a987129addb022a330f7f89b672eca"
server_key_version: "encryptionkey@0"
I20260812 14:21:52.845256 32517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:52.870998 32517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:52.873750 32517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:52.873939 32517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:52.879218 32517 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42951
I20260812 14:21:52.879251 32599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42951 every 8 connection(s)
I20260812 14:21:52.879667 32517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:52.880735 32600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:52.883077 32600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe: Bootstrap starting.
I20260812 14:21:52.883745 32600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:52.884042 32600 log.cc:824] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:52.884749 32600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe: No bootstrap required, opened a new log
I20260812 14:21:52.886380 32600 raft_consensus.cc:348] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b251345ce1b94dfcba0f1b504952a4fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42951 } }
I20260812 14:21:52.886515 32600 raft_consensus.cc:374] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:52.886562 32600 raft_consensus.cc:729] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b251345ce1b94dfcba0f1b504952a4fe, State: Initialized, Role: FOLLOWER
I20260812 14:21:52.886749 32600 consensus_queue.cc:260] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b251345ce1b94dfcba0f1b504952a4fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42951 } }
I20260812 14:21:52.886835 32600 raft_consensus.cc:388] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:52.886858 32600 raft_consensus.cc:482] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:52.886909 32600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:52.887482 32600 raft_consensus.cc:504] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b251345ce1b94dfcba0f1b504952a4fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42951 } }
I20260812 14:21:52.887600 32600 leader_election.cc:302] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b251345ce1b94dfcba0f1b504952a4fe; no voters:
I20260812 14:21:52.887759 32600 leader_election.cc:288] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:21:52.887820 32604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:52.888031 32600 sys_catalog.cc:564] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:52.888052 32604 raft_consensus.cc:686] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [term 1 LEADER]: Becoming Leader. State: Replica: b251345ce1b94dfcba0f1b504952a4fe, State: Running, Role: LEADER
I20260812 14:21:52.888163 32604 consensus_queue.cc:237] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b251345ce1b94dfcba0f1b504952a4fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42951 } }
I20260812 14:21:52.888734 32606 sys_catalog.cc:455] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b251345ce1b94dfcba0f1b504952a4fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b251345ce1b94dfcba0f1b504952a4fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42951 } } }
I20260812 14:21:52.888856 32606 sys_catalog.cc:458] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:52.888734 32607 sys_catalog.cc:455] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader b251345ce1b94dfcba0f1b504952a4fe. Latest consensus state: current_term: 1 leader_uuid: "b251345ce1b94dfcba0f1b504952a4fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b251345ce1b94dfcba0f1b504952a4fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42951 } } }
I20260812 14:21:52.888980 32607 sys_catalog.cc:458] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:52.888996 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 32517
I20260812 14:21:52.889111 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:21:52.889308 8532 external_mini_cluster.cc:1428] Setting key 62ad6b41a60d3d8103f8bbbe05a5ad6b
I20260812 14:21:52.889951 32615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:52.891714 32615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:52.893342 32615 catalog_manager.cc:1347] Generated new cluster ID: a918f216a57a4d0396899a095c61656b
I20260812 14:21:52.893411 32615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:21:52 dist-test-slave-5k54 krb5kdc[32501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 14:21:52.900784 32624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:52.889820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56087 (local address 127.0.0.1:42951)
0812 14:21:52.890532 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:52.890540 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:52.890566 (+ 26us) server_negotiation.cc:407] Connection header received
0812 14:21:52.890660 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:52.890667 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:52.890709 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:52.890820 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:52.891637 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:52.892824 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:52.893791 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:52.894232 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:52.894351 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:52.896745 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:52.896749 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:52.896749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:52.896761 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:52.898639 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:52.899057 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:52.899060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:52.899061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:52.899191 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:52.899449 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:52.899450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:52.899451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:52.899789 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:52.899955 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:52.900417 (+ 462us) server_negotiation.cc:299] Negotiation successful
0812 14:21:52.900565 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":94,"threads_started":1}
I20260812 14:21:52.902110 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:21:52.913558 32624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:52.903481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38117 (local address 127.0.0.1:42951)
0812 14:21:52.903519 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:52.903521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:52.907609 (+ 4088us) server_negotiation.cc:407] Connection header received
0812 14:21:52.907683 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:52.907684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:52.907695 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:52.907753 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:52.908261 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:52.908751 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:52.909521 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:52.909735 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:52.909820 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:52.910990 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:52.910992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:52.910992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:52.910999 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:52.912403 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:52.912768 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:52.912769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:52.912769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:52.912831 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:52.913053 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:52.913054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:52.913054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:52.913166 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:52.913283 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:52.913378 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 14:21:52.913455 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:21:52.914644 32615 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:21:52.915314 32615 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:52.920249 32615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b251345ce1b94dfcba0f1b504952a4fe: Generated new TSK 0
I20260812 14:21:52.920416 32615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:21:52.925062 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 32517
2026-08-12T14:21:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:21:52 dist-test-slave-5k54 krb5kdc[32637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:21:52 dist-test-slave-5k54 krb5kdc[32637](info): set up 2 sockets
Aug 12 14:21:52 dist-test-slave-5k54 krb5kdc[32637](info): commencing operation
krb5kdc: starting...
W20260812 14:21:54.045419 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:21:54 dist-test-slave-5k54 krb5kdc[32637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:21:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:21:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:21:54.173882 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44341
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:54.256211 32652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:54.256466 32652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:54.256507 32652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:54.257665 32652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:54.257730 32652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:54.257755 32652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:54.257782 32652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:54.257815 32652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:54.259294 32652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:54.259598 32652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:54.259829 32652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:54.264807 32661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:54.264964 32652 server_base.cc:1034] running on GCE node
W20260812 14:21:54.264823 32659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:54.264818 32657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:54.265322 32652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:54.265638 32652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:54.266827 32652 hybrid_clock.cc:648] HybridClock initialized: now 1786544514266803 us; error 35 us; skew 500 ppm
Aug 12 14:21:54 dist-test-slave-5k54 krb5kdc[32637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544514, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:54.269444 32652 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:21:54.269866 32652 webserver.cc:460] Webserver started at http://127.0.0.1:38931/ using document root <none> and password file <none>
I20260812 14:21:54.270116 32652 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:54.270188 32652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:54.270336 32652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:54.271391 32652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "96b5e64eaa64497d89f336947adf8335"
format_stamp: "Formatted at 2026-08-12 14:21:54 on dist-test-slave-5k54"
server_key: "3260e8b9e45fe1de7e4a5755a0769ec4"
server_key_iv: "2319e3e7271b8522ce6d272d77a9675f"
server_key_version: "encryptionkey@0"
I20260812 14:21:54.271829 32652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "96b5e64eaa64497d89f336947adf8335"
format_stamp: "Formatted at 2026-08-12 14:21:54 on dist-test-slave-5k54"
server_key: "3260e8b9e45fe1de7e4a5755a0769ec4"
server_key_iv: "2319e3e7271b8522ce6d272d77a9675f"
server_key_version: "encryptionkey@0"
I20260812 14:21:54.273137 32652 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:21:54.273861 32675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:54.274053 32652 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:21:54.274163 32652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "96b5e64eaa64497d89f336947adf8335"
format_stamp: "Formatted at 2026-08-12 14:21:54 on dist-test-slave-5k54"
server_key: "3260e8b9e45fe1de7e4a5755a0769ec4"
server_key_iv: "2319e3e7271b8522ce6d272d77a9675f"
server_key_version: "encryptionkey@0"
I20260812 14:21:54.274240 32652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:54.288228 32652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:54.290508 32652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:54.290679 32652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:54.294988 32652 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44341
I20260812 14:21:54.295029 32741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44341 every 8 connection(s)
I20260812 14:21:54.295398 32652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:54.296402 32742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:54.298519 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 32652
I20260812 14:21:54.298622 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:21:54.298776 32742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335: Bootstrap starting.
I20260812 14:21:54.298834 8532 external_mini_cluster.cc:1428] Setting key 184ac293ce75cbf454607d7f8a5cb4ee
I20260812 14:21:54.299512 32742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:54.299820 32742 log.cc:824] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:54.300576 32742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335: No bootstrap required, opened a new log
I20260812 14:21:54.302711 32742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b5e64eaa64497d89f336947adf8335" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44341 } }
I20260812 14:21:54.302846 32742 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:54.302871 32742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b5e64eaa64497d89f336947adf8335, State: Initialized, Role: FOLLOWER
I20260812 14:21:54.303018 32742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b5e64eaa64497d89f336947adf8335" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44341 } }
I20260812 14:21:54.303078 32742 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:54.303099 32742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:54.303165 32742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:54.303773 32742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b5e64eaa64497d89f336947adf8335" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44341 } }
I20260812 14:21:54.303894 32742 leader_election.cc:302] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96b5e64eaa64497d89f336947adf8335; no voters:
I20260812 14:21:54.304095 32742 leader_election.cc:288] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:21:54.304131 32748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:54.304303 32748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [term 1 LEADER]: Becoming Leader. State: Replica: 96b5e64eaa64497d89f336947adf8335, State: Running, Role: LEADER
I20260812 14:21:54.304359 32742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:54.304438 32748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b5e64eaa64497d89f336947adf8335" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44341 } }
I20260812 14:21:54.304821 32748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96b5e64eaa64497d89f336947adf8335. Latest consensus state: current_term: 1 leader_uuid: "96b5e64eaa64497d89f336947adf8335" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b5e64eaa64497d89f336947adf8335" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44341 } } }
I20260812 14:21:54.304833 32751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96b5e64eaa64497d89f336947adf8335" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96b5e64eaa64497d89f336947adf8335" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44341 } } }
I20260812 14:21:54.304837 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:54.299358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39737 (local address 127.0.0.1:44341)
0812 14:21:54.299595 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:54.299604 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:54.299635 (+ 31us) server_negotiation.cc:407] Connection header received
0812 14:21:54.300118 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:54.300126 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:54.300176 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:54.300292 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:54.300540 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:54.301455 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:54.302639 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:54.303202 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:54.303319 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:54.304001 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:54.304005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:54.304005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:54.304222 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:54.304261 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:54.304276 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:54.304532 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:54.304565 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 14:21:54.304632 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":132,"threads_started":1}
I20260812 14:21:54.305274 32751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:54.305639 32748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:54.305685 32761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:54.305989 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:21:54.306177 32761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:54.307888 32761 catalog_manager.cc:1347] Generated new cluster ID: d9b81c1858fc44b7a83a6f4b46a1b42f
I20260812 14:21:54.307942 32761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:21:54.313995 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:54.307439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34541 (local address 127.0.0.1:44341)
0812 14:21:54.307479 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:54.307481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:54.307599 (+ 118us) server_negotiation.cc:407] Connection header received
0812 14:21:54.307628 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:54.307629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:54.307639 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:54.307709 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:54.311544 (+ 3835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:54.312046 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:54.312791 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:54.313010 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:54.313088 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:54.313744 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:54.313746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:54.313746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:54.313751 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:54.313765 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:54.313769 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:54.313812 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:54.313872 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 14:21:54.313898 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:21:54.317145 32761 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:21:54.317853 32761 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:54.323079 32761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96b5e64eaa64497d89f336947adf8335: Generated new TSK 0
I20260812 14:21:54.323266 32761 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:21:54.325882 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 32652
2026-08-12T14:21:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1400 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:21:54 dist-test-slave-5k54 krb5kdc[311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:21:54 dist-test-slave-5k54 krb5kdc[311](info): set up 2 sockets
Aug 12 14:21:54 dist-test-slave-5k54 krb5kdc[311](info): commencing operation
krb5kdc: starting...
W20260812 14:21:55.458377 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:21:55 dist-test-slave-5k54 krb5kdc[311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:21:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:21:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:21:55.587091 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36601
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:55.670828 325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:55.671038 325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:55.671061 325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:55.672205 325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:55.672250 325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:55.672263 325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:55.672276 325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:55.672288 325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:55.673671 325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:55.673846 325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:55.674023 325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:55.679042 333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:55.679054 331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:55.679114 330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:55.679360 325 server_base.cc:1034] running on GCE node
I20260812 14:21:55.679630 325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:55.679970 325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:55.681147 325 hybrid_clock.cc:648] HybridClock initialized: now 1786544515681132 us; error 49 us; skew 500 ppm
Aug 12 14:21:55 dist-test-slave-5k54 krb5kdc[311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544515, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:55.683971 325 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:21:55.684460 325 webserver.cc:460] Webserver started at http://127.0.0.1:40197/ using document root <none> and password file <none>
I20260812 14:21:55.684763 325 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:55.684856 325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:55.685034 325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:55.686201 325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "c6a1abccfd064de3bdcc054f7396de46"
format_stamp: "Formatted at 2026-08-12 14:21:55 on dist-test-slave-5k54"
server_key: "0c676a30e5dbbfff4ce18a90e384a0aa"
server_key_iv: "0b6dc9272b75ee58c75813d1d1a3c625"
server_key_version: "encryptionkey@0"
I20260812 14:21:55.686628 325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "c6a1abccfd064de3bdcc054f7396de46"
format_stamp: "Formatted at 2026-08-12 14:21:55 on dist-test-slave-5k54"
server_key: "0c676a30e5dbbfff4ce18a90e384a0aa"
server_key_iv: "0b6dc9272b75ee58c75813d1d1a3c625"
server_key_version: "encryptionkey@0"
I20260812 14:21:55.688215 325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:21:55.689169 340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:55.689437 325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:21:55.689561 325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "c6a1abccfd064de3bdcc054f7396de46"
format_stamp: "Formatted at 2026-08-12 14:21:55 on dist-test-slave-5k54"
server_key: "0c676a30e5dbbfff4ce18a90e384a0aa"
server_key_iv: "0b6dc9272b75ee58c75813d1d1a3c625"
server_key_version: "encryptionkey@0"
I20260812 14:21:55.689647 325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:55.693538 325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:55.695806 325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:55.695976 325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:55.700584 325 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36601
I20260812 14:21:55.700613 400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36601 every 8 connection(s)
I20260812 14:21:55.700985 325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:55.701999 402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:55.702550 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 325
I20260812 14:21:55.702696 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:21:55.702908 8532 external_mini_cluster.cc:1428] Setting key 264d401acff195d566cba0bac9ae8a80
I20260812 14:21:55.705160 402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46: Bootstrap starting.
I20260812 14:21:55.705686 405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:55.703492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58067 (local address 127.0.0.1:36601)
0812 14:21:55.703683 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:55.703693 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:55.703727 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:21:55.704319 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:55.704329 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:55.704366 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:55.704520 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:55.705247 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:55.705250 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:55.705250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:55.705255 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:55.705299 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:55.705320 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:55.705390 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:55.705443 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 14:21:55.705484 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260812 14:21:55.705904 402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:55.706233 402 log.cc:824] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:55.706787 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:21:55.707069 402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46: No bootstrap required, opened a new log
I20260812 14:21:55.709300 405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:55.708343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42441 (local address 127.0.0.1:36601)
0812 14:21:55.708381 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:55.708382 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:55.708480 (+ 98us) server_negotiation.cc:407] Connection header received
0812 14:21:55.708534 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:55.708534 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:55.708538 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:55.708582 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:55.709105 (+ 523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:55.709106 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:55.709106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:55.709110 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:55.709120 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:55.709123 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:55.709162 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:55.709214 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 14:21:55.709223 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:21:55.709450 402 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a1abccfd064de3bdcc054f7396de46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36601 } }
I20260812 14:21:55.709617 402 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:55.709661 402 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6a1abccfd064de3bdcc054f7396de46, State: Initialized, Role: FOLLOWER
I20260812 14:21:55.709784 402 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a1abccfd064de3bdcc054f7396de46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36601 } }
I20260812 14:21:55.709879 402 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:55.709903 402 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:55.709973 402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:55.710490 402 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a1abccfd064de3bdcc054f7396de46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36601 } }
I20260812 14:21:55.710602 402 leader_election.cc:302] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6a1abccfd064de3bdcc054f7396de46; no voters:
I20260812 14:21:55.710783 402 leader_election.cc:288] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:21:55.710896 413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:55.711035 402 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:55.711064 413 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [term 1 LEADER]: Becoming Leader. State: Replica: c6a1abccfd064de3bdcc054f7396de46, State: Running, Role: LEADER
I20260812 14:21:55.711145 413 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a1abccfd064de3bdcc054f7396de46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36601 } }
I20260812 14:21:55.711509 414 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6a1abccfd064de3bdcc054f7396de46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a1abccfd064de3bdcc054f7396de46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36601 } } }
I20260812 14:21:55.711633 414 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:55.711520 415 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6a1abccfd064de3bdcc054f7396de46. Latest consensus state: current_term: 1 leader_uuid: "c6a1abccfd064de3bdcc054f7396de46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6a1abccfd064de3bdcc054f7396de46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36601 } } }
I20260812 14:21:55.711902 415 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46 [sys.catalog]: This master's current role is: LEADER
W20260812 14:21:55.712850 434 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:21:55.712922 434 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:21:55.713016 422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:55.713192 422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:55.714691 422 catalog_manager.cc:1347] Generated new cluster ID: d60433c812054d4892cd2e2d33e8c28d
I20260812 14:21:55.714744 422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:21:55.728243 422 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:21:55.728426 422 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:55.738672 422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6a1abccfd064de3bdcc054f7396de46: Generated new TSK 0
I20260812 14:21:55.738888 422 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:21:55.744792 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 325
2026-08-12T14:21:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1414 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:21:55 dist-test-slave-5k54 krb5kdc[440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:21:55 dist-test-slave-5k54 krb5kdc[440](info): set up 2 sockets
Aug 12 14:21:55 dist-test-slave-5k54 krb5kdc[440](info): commencing operation
krb5kdc: starting...
W20260812 14:21:56.859892 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:21:56 dist-test-slave-5k54 krb5kdc[440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:21:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:21:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:21:56.989830 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33481
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:57.075603 454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:57.075804 454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:57.075829 454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:57.077024 454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:57.077070 454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:57.077085 454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:57.077097 454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:57.077143 454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:57.078608 454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:57.078853 454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:57.079072 454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:57.083841 462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:57.083841 460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:57.083997 459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:57.084218 454 server_base.cc:1034] running on GCE node
I20260812 14:21:57.084458 454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:57.084698 454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:57.085850 454 hybrid_clock.cc:648] HybridClock initialized: now 1786544517085822 us; error 39 us; skew 500 ppm
Aug 12 14:21:57 dist-test-slave-5k54 krb5kdc[440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544517, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:57.094157 454 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:21:57.094661 454 webserver.cc:460] Webserver started at http://127.0.0.1:40443/ using document root <none> and password file <none>
I20260812 14:21:57.094897 454 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:57.094981 454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:57.095119 454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:57.096215 454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "03eeb458b13f40389262275b6fff4b33"
format_stamp: "Formatted at 2026-08-12 14:21:57 on dist-test-slave-5k54"
server_key: "a3c6c9b16557b88c33bf9cf57ab8676e"
server_key_iv: "6bd8ef1cfdfcf0c680caddf25b16b26d"
server_key_version: "encryptionkey@0"
I20260812 14:21:57.096537 454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "03eeb458b13f40389262275b6fff4b33"
format_stamp: "Formatted at 2026-08-12 14:21:57 on dist-test-slave-5k54"
server_key: "a3c6c9b16557b88c33bf9cf57ab8676e"
server_key_iv: "6bd8ef1cfdfcf0c680caddf25b16b26d"
server_key_version: "encryptionkey@0"
I20260812 14:21:57.097815 454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:21:57.098505 470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:57.098712 454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:21:57.098829 454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "03eeb458b13f40389262275b6fff4b33"
format_stamp: "Formatted at 2026-08-12 14:21:57 on dist-test-slave-5k54"
server_key: "a3c6c9b16557b88c33bf9cf57ab8676e"
server_key_iv: "6bd8ef1cfdfcf0c680caddf25b16b26d"
server_key_version: "encryptionkey@0"
I20260812 14:21:57.098910 454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:57.109189 454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:57.111223 454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:57.111399 454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:57.115746 454 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33481
I20260812 14:21:57.115782 527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33481 every 8 connection(s)
I20260812 14:21:57.116148 454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:57.117173 528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:57.119577 528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33: Bootstrap starting.
I20260812 14:21:57.120144 528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:57.120389 528 log.cc:824] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:57.121062 528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33: No bootstrap required, opened a new log
I20260812 14:21:57.122613 528 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03eeb458b13f40389262275b6fff4b33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } }
I20260812 14:21:57.122754 528 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:57.122777 528 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03eeb458b13f40389262275b6fff4b33, State: Initialized, Role: FOLLOWER
I20260812 14:21:57.122880 528 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03eeb458b13f40389262275b6fff4b33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } }
I20260812 14:21:57.122936 528 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:57.122956 528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:57.123035 528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:57.123586 528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03eeb458b13f40389262275b6fff4b33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } }
I20260812 14:21:57.123708 528 leader_election.cc:302] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03eeb458b13f40389262275b6fff4b33; no voters:
I20260812 14:21:57.123893 528 leader_election.cc:288] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:21:57.123936 532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:57.124121 532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [term 1 LEADER]: Becoming Leader. State: Replica: 03eeb458b13f40389262275b6fff4b33, State: Running, Role: LEADER
I20260812 14:21:57.124161 528 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:57.124243 532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03eeb458b13f40389262275b6fff4b33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } }
I20260812 14:21:57.124727 535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03eeb458b13f40389262275b6fff4b33. Latest consensus state: current_term: 1 leader_uuid: "03eeb458b13f40389262275b6fff4b33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03eeb458b13f40389262275b6fff4b33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } } }
I20260812 14:21:57.124825 535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:57.124897 533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03eeb458b13f40389262275b6fff4b33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03eeb458b13f40389262275b6fff4b33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33481 } } }
I20260812 14:21:57.124987 533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:57.125298 542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:57.125705 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 454
I20260812 14:21:57.125823 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:21:57.125846 542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:57.126019 8532 external_mini_cluster.cc:1428] Setting key 89ece39b4f7d92a61995b6df50924d44
I20260812 14:21:57.128098 542 catalog_manager.cc:1347] Generated new cluster ID: b01daa4aa8ca415d910b3757d49945c5
I20260812 14:21:57.128154 542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:21:57 dist-test-slave-5k54 krb5kdc[440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 14:21:57.137846 542 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:21:57.138514 542 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:57.138875 554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:57.126602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:42847 (local address 127.0.0.1:33481)
0812 14:21:57.127137 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:57.127147 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:57.127178 (+ 31us) server_negotiation.cc:407] Connection header received
0812 14:21:57.127290 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:57.127298 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:57.127355 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:57.127518 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:57.128454 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:57.130027 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:57.131076 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:57.131783 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:57.131919 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:57.134840 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:57.134844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:57.134845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:57.134857 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:57.136878 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:57.137355 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:57.137358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:57.137358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:57.137462 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:57.137894 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:57.137895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:57.137895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:57.138170 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:57.138319 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:57.138504 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 14:21:57.138617 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":98,"threads_started":1}
I20260812 14:21:57.139858 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:21:57.145936 542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03eeb458b13f40389262275b6fff4b33: Generated new TSK 0
I20260812 14:21:57.146150 542 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:21:57.148717 554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:57.141378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:55859 (local address 127.0.0.1:33481)
0812 14:21:57.141420 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:57.141422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:57.141571 (+ 149us) server_negotiation.cc:407] Connection header received
0812 14:21:57.141627 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:57.141628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:57.141648 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:57.141698 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:57.142293 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:57.143108 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:57.144186 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:57.144495 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:57.144558 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:57.146044 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:57.146045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:57.146045 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:21:57.146052 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:57.147476 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:57.147918 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:57.147919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:57.147919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:57.147968 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:21:57.148245 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:21:57.148247 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:21:57.148247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:21:57.148357 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:57.148450 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:57.148610 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 14:21:57.148663 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:21:57.150909 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 454
W20260812 14:21:57.158701 438 connection.cc:535] client connection to 127.0.0.1:33481 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33481): Connection reset by peer (error 104)
2026-08-12T14:21:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:21:57 dist-test-slave-5k54 krb5kdc[563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:21:57 dist-test-slave-5k54 krb5kdc[563](info): set up 2 sockets
Aug 12 14:21:57 dist-test-slave-5k54 krb5kdc[563](info): commencing operation
krb5kdc: starting...
W20260812 14:21:58.274160 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:21:58 dist-test-slave-5k54 krb5kdc[563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:21:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:21:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:21:58.402288 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35407
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:58.484936 579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:58.485136 579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:58.485158 579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:58.486292 579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:58.486337 579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:58.486352 579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:58.486366 579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:58.486377 579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:58.487975 579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:58.488230 579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:58.488476 579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:58.493736 590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:58.493746 586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:58.494055 579 server_base.cc:1034] running on GCE node
W20260812 14:21:58.493738 587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:58.494390 579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:58.494692 579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:58.495880 579 hybrid_clock.cc:648] HybridClock initialized: now 1786544518495876 us; error 38 us; skew 500 ppm
Aug 12 14:21:58 dist-test-slave-5k54 krb5kdc[563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544518, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:58.498365 579 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:21:58.498785 579 webserver.cc:460] Webserver started at http://127.0.0.1:39151/ using document root <none> and password file <none>
I20260812 14:21:58.499068 579 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:58.499141 579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:58.499287 579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:58.500383 579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "aa6ec11602b440b8bbeb5b3ed346d2d1"
format_stamp: "Formatted at 2026-08-12 14:21:58 on dist-test-slave-5k54"
server_key: "3aa1070d7a1afabe828b888c755a3a5b"
server_key_iv: "08b2bb998ac190869e4d01dd939cc5c7"
server_key_version: "encryptionkey@0"
I20260812 14:21:58.500754 579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "aa6ec11602b440b8bbeb5b3ed346d2d1"
format_stamp: "Formatted at 2026-08-12 14:21:58 on dist-test-slave-5k54"
server_key: "3aa1070d7a1afabe828b888c755a3a5b"
server_key_iv: "08b2bb998ac190869e4d01dd939cc5c7"
server_key_version: "encryptionkey@0"
I20260812 14:21:58.502092 579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:21:58.502810 598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:58.502993 579 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:21:58.503093 579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "aa6ec11602b440b8bbeb5b3ed346d2d1"
format_stamp: "Formatted at 2026-08-12 14:21:58 on dist-test-slave-5k54"
server_key: "3aa1070d7a1afabe828b888c755a3a5b"
server_key_iv: "08b2bb998ac190869e4d01dd939cc5c7"
server_key_version: "encryptionkey@0"
I20260812 14:21:58.503173 579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:58.516737 579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:58.518642 579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:58.518814 579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:58.522958 579 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35407
I20260812 14:21:58.523125 655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35407 every 8 connection(s)
I20260812 14:21:58.523406 579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:58.524500 656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:58.527087 656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1: Bootstrap starting.
I20260812 14:21:58.527369 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 579
I20260812 14:21:58.527534 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:21:58.527740 8532 external_mini_cluster.cc:1428] Setting key 108b2d275030d094a8a1a2a65f701071
I20260812 14:21:58.527912 656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:58.528262 656 log.cc:824] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:58.529115 656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1: No bootstrap required, opened a new log
I20260812 14:21:58.531114 656 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa6ec11602b440b8bbeb5b3ed346d2d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35407 } }
I20260812 14:21:58.531301 656 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:58.531347 656 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa6ec11602b440b8bbeb5b3ed346d2d1, State: Initialized, Role: FOLLOWER
I20260812 14:21:58.531524 656 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa6ec11602b440b8bbeb5b3ed346d2d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35407 } }
I20260812 14:21:58.531641 656 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:58.531693 656 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:58.531749 656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:58.532526 656 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa6ec11602b440b8bbeb5b3ed346d2d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35407 } }
I20260812 14:21:58.532672 656 leader_election.cc:302] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa6ec11602b440b8bbeb5b3ed346d2d1; no voters:
I20260812 14:21:58.532881 656 leader_election.cc:288] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:21:58.533080 662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:58.533172 656 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:58.533232 662 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [term 1 LEADER]: Becoming Leader. State: Replica: aa6ec11602b440b8bbeb5b3ed346d2d1, State: Running, Role: LEADER
I20260812 14:21:58.533766 659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:58.528411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:51253 (local address 127.0.0.1:35407)
0812 14:21:58.528631 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:58.528643 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:58.528677 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:21:58.529209 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:58.529220 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:58.529294 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:58.529455 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:58.529684 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:58.530587 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:58.531430 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:58.531995 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:58.532081 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:58.532926 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:58.532930 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:58.532931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:58.533180 (+ 249us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:58.533224 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:58.533240 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:58.533352 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:58.533521 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 14:21:58.533554 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":90,"threads_started":1}
I20260812 14:21:58.533882 662 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa6ec11602b440b8bbeb5b3ed346d2d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35407 } }
I20260812 14:21:58.534241 661 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa6ec11602b440b8bbeb5b3ed346d2d1. Latest consensus state: current_term: 1 leader_uuid: "aa6ec11602b440b8bbeb5b3ed346d2d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa6ec11602b440b8bbeb5b3ed346d2d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35407 } } }
I20260812 14:21:58.534345 661 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:58.534694 663 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa6ec11602b440b8bbeb5b3ed346d2d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa6ec11602b440b8bbeb5b3ed346d2d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35407 } } }
I20260812 14:21:58.534981 663 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:58.535157 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 14:21:58.535194 678 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:21:58.535260 678 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:21:58.535351 675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:58.535573 675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:58.537563 675 catalog_manager.cc:1347] Generated new cluster ID: 8f5347a1a1f64dacb2614f2c70a69fff
I20260812 14:21:58.537617 675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:21:58.543318 659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:58.536799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:34435 (local address 127.0.0.1:35407)
0812 14:21:58.536839 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:58.536841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:58.539636 (+ 2795us) server_negotiation.cc:407] Connection header received
0812 14:21:58.539701 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:58.539702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:58.539712 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:58.539766 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:58.539941 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:58.540653 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:58.541888 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:21:58.542200 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:21:58.542256 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:21:58.543051 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:58.543052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:58.543052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:58.543057 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:58.543072 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:58.543076 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:58.543124 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:58.543238 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 14:21:58.543247 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:21:58.549921 675 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:21:58.550449 675 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:58.557662 675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa6ec11602b440b8bbeb5b3ed346d2d1: Generated new TSK 0
I20260812 14:21:58.557840 675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:21:58.562570 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 579
W20260812 14:21:58.570911 561 connection.cc:535] client connection to 127.0.0.1:35407 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35407): Connection reset by peer (error 104)
2026-08-12T14:21:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:21:58 dist-test-slave-5k54 krb5kdc[690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:21:58 dist-test-slave-5k54 krb5kdc[690](info): set up 2 sockets
Aug 12 14:21:58 dist-test-slave-5k54 krb5kdc[690](info): commencing operation
krb5kdc: starting...
W20260812 14:21:59.687258 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:21:59 dist-test-slave-5k54 krb5kdc[690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:21:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:21:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:21:59.822726 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36297
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:21:59.906596 704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:21:59.906800 704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:21:59.906823 704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:21:59.907913 704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:21:59.907951 704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:21:59.907966 704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:21:59.907979 704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:21:59.907990 704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:21:59.909382 704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:21:59.909570 704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:21:59.909732 704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:21:59.914624 712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:59.914605 710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:21:59.914600 709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:21:59.914803 704 server_base.cc:1034] running on GCE node
I20260812 14:21:59.915174 704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:21:59.915498 704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:21:59.916653 704 hybrid_clock.cc:648] HybridClock initialized: now 1786544519916633 us; error 38 us; skew 500 ppm
Aug 12 14:21:59 dist-test-slave-5k54 krb5kdc[690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544519, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:21:59.919205 704 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:21:59.919685 704 webserver.cc:460] Webserver started at http://127.0.0.1:37625/ using document root <none> and password file <none>
I20260812 14:21:59.919910 704 fs_manager.cc:359] Metadata directory not provided
I20260812 14:21:59.919955 704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:21:59.920045 704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:21:59.921027 704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "e6eb0f37f22d4f199a6ea4665dc33b57"
format_stamp: "Formatted at 2026-08-12 14:21:59 on dist-test-slave-5k54"
server_key: "35c76599aef5c622b06f0a0ec9e26ffe"
server_key_iv: "604cc677227f7b0e8a3d40f077942ced"
server_key_version: "encryptionkey@0"
I20260812 14:21:59.921407 704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "e6eb0f37f22d4f199a6ea4665dc33b57"
format_stamp: "Formatted at 2026-08-12 14:21:59 on dist-test-slave-5k54"
server_key: "35c76599aef5c622b06f0a0ec9e26ffe"
server_key_iv: "604cc677227f7b0e8a3d40f077942ced"
server_key_version: "encryptionkey@0"
I20260812 14:21:59.922675 704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:21:59.923364 720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:21:59.923586 704 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:21:59.923686 704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "e6eb0f37f22d4f199a6ea4665dc33b57"
format_stamp: "Formatted at 2026-08-12 14:21:59 on dist-test-slave-5k54"
server_key: "35c76599aef5c622b06f0a0ec9e26ffe"
server_key_iv: "604cc677227f7b0e8a3d40f077942ced"
server_key_version: "encryptionkey@0"
I20260812 14:21:59.923779 704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:21:59.927322 704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:21:59.930030 704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:21:59.930137 704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:21:59.934098 704 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36297
I20260812 14:21:59.934128 781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36297 every 8 connection(s)
I20260812 14:21:59.934459 704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:21:59.935549 782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:21:59.937798 782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57: Bootstrap starting.
I20260812 14:21:59.938413 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 704
I20260812 14:21:59.938428 782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57: Neither blocks nor log segments found. Creating new log.
I20260812 14:21:59.938547 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:21:59.938722 8532 external_mini_cluster.cc:1428] Setting key 1fed4fb384dfec089a452024e3c845d4
I20260812 14:21:59.938762 782 log.cc:824] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57: Log is configured to *not* fsync() on all Append() calls
I20260812 14:21:59.939695 782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57: No bootstrap required, opened a new log
I20260812 14:21:59.941468 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:59.939286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:34905 (local address 127.0.0.1:36297)
0812 14:21:59.939502 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:59.939511 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:59.939652 (+ 141us) server_negotiation.cc:407] Connection header received
0812 14:21:59.940192 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:59.940201 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:59.940236 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:59.940400 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:59.941020 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:59.941023 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:59.941024 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:59.941028 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:59.941074 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:59.941089 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:59.941190 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:59.941221 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 14:21:59.941263 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":58,"threads_started":1}
I20260812 14:21:59.942559 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:21:59.942466 782 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6eb0f37f22d4f199a6ea4665dc33b57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36297 } }
I20260812 14:21:59.942628 782 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:21:59.942660 782 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6eb0f37f22d4f199a6ea4665dc33b57, State: Initialized, Role: FOLLOWER
I20260812 14:21:59.942771 782 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6eb0f37f22d4f199a6ea4665dc33b57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36297 } }
I20260812 14:21:59.942849 782 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:21:59.942894 782 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:21:59.942937 782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:21:59.943679 782 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6eb0f37f22d4f199a6ea4665dc33b57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36297 } }
I20260812 14:21:59.943811 782 leader_election.cc:302] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6eb0f37f22d4f199a6ea4665dc33b57; no voters:
I20260812 14:21:59.943994 782 leader_election.cc:288] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:21:59.944031 792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:21:59.944185 792 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [term 1 LEADER]: Becoming Leader. State: Replica: e6eb0f37f22d4f199a6ea4665dc33b57, State: Running, Role: LEADER
I20260812 14:21:59.944237 782 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:21:59.944319 792 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6eb0f37f22d4f199a6ea4665dc33b57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36297 } }
I20260812 14:21:59.944698 792 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6eb0f37f22d4f199a6ea4665dc33b57. Latest consensus state: current_term: 1 leader_uuid: "e6eb0f37f22d4f199a6ea4665dc33b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6eb0f37f22d4f199a6ea4665dc33b57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36297 } } }
I20260812 14:21:59.944715 793 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6eb0f37f22d4f199a6ea4665dc33b57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6eb0f37f22d4f199a6ea4665dc33b57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36297 } } }
I20260812 14:21:59.944818 792 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:59.944842 793 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57 [sys.catalog]: This master's current role is: LEADER
I20260812 14:21:59.945555 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:21:59.944310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:50273 (local address 127.0.0.1:36297)
0812 14:21:59.944348 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:21:59.944350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:21:59.944445 (+ 95us) server_negotiation.cc:407] Connection header received
0812 14:21:59.944543 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:21:59.944544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:21:59.944548 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:21:59.944579 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:21:59.945268 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:21:59.945270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:21:59.945270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:21:59.945275 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:21:59.945288 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:21:59.945293 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:21:59.945345 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:21:59.945475 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 14:21:59.945491 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 14:21:59.946184 811 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:21:59.946230 811 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:21:59.946326 803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:21:59.946471 803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:21:59.947856 803 catalog_manager.cc:1347] Generated new cluster ID: e35dc51b85164e12a21bc35e36cbd551
I20260812 14:21:59.947911 803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:21:59.964488 803 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:21:59.964658 803 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:21:59.968227 803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6eb0f37f22d4f199a6ea4665dc33b57: Generated new TSK 0
I20260812 14:21:59.968433 803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:21:59.970731 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 704
2026-08-12T14:21:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:22:00 dist-test-slave-5k54 krb5kdc[816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:22:00 dist-test-slave-5k54 krb5kdc[816](info): set up 2 sockets
Aug 12 14:22:00 dist-test-slave-5k54 krb5kdc[816](info): commencing operation
krb5kdc: starting...
W20260812 14:22:01.101814 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:22:01 dist-test-slave-5k54 krb5kdc[816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:22:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:22:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:22:01.224596 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40751
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:22:01.308037 836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:22:01.308244 836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:22:01.308267 836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:22:01.309337 836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:22:01.309370 836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:22:01.309386 836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:22:01.309397 836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:22:01.309409 836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:22:01.310789 836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40751
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:22:01.310974 836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:22:01.311138 836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:22:01.315970 841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:01.316040 842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:01.315971 844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:01.316344 836 server_base.cc:1034] running on GCE node
I20260812 14:22:01.316542 836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:22:01.316819 836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:22:01.317968 836 hybrid_clock.cc:648] HybridClock initialized: now 1786544521317948 us; error 34 us; skew 500 ppm
Aug 12 14:22:01 dist-test-slave-5k54 krb5kdc[816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544521, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:22:01.320564 836 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:22:01.321048 836 webserver.cc:460] Webserver started at http://127.0.0.1:40661/ using document root <none> and password file <none>
I20260812 14:22:01.321275 836 fs_manager.cc:359] Metadata directory not provided
I20260812 14:22:01.321342 836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:22:01.321499 836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:22:01.322566 836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "9f85fdc722e64753aac27ec3fc351894"
format_stamp: "Formatted at 2026-08-12 14:22:01 on dist-test-slave-5k54"
server_key: "a30382d7e7d4eecbde620c9e439d0d63"
server_key_iv: "34db3b4c1122f9f4543d787a0712fa3f"
server_key_version: "encryptionkey@0"
I20260812 14:22:01.322952 836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "9f85fdc722e64753aac27ec3fc351894"
format_stamp: "Formatted at 2026-08-12 14:22:01 on dist-test-slave-5k54"
server_key: "a30382d7e7d4eecbde620c9e439d0d63"
server_key_iv: "34db3b4c1122f9f4543d787a0712fa3f"
server_key_version: "encryptionkey@0"
I20260812 14:22:01.324360 836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:22:01.325114 852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:01.325284 836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:01.325387 836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "9f85fdc722e64753aac27ec3fc351894"
format_stamp: "Formatted at 2026-08-12 14:22:01 on dist-test-slave-5k54"
server_key: "a30382d7e7d4eecbde620c9e439d0d63"
server_key_iv: "34db3b4c1122f9f4543d787a0712fa3f"
server_key_version: "encryptionkey@0"
I20260812 14:22:01.325470 836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:22:01.337091 836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:22:01.339344 836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:22:01.339583 836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:22:01.343813 836 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40751
I20260812 14:22:01.343916 913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40751 every 8 connection(s)
I20260812 14:22:01.344233 836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:22:01.345230 914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:22:01.347558 914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894: Bootstrap starting.
I20260812 14:22:01.348137 914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894: Neither blocks nor log segments found. Creating new log.
I20260812 14:22:01.348445 914 log.cc:824] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894: Log is configured to *not* fsync() on all Append() calls
I20260812 14:22:01.349141 914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894: No bootstrap required, opened a new log
I20260812 14:22:01.349784 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 836
I20260812 14:22:01.349889 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:22:01.350144 8532 external_mini_cluster.cc:1428] Setting key 8929a8fdcdfec4e1f44826b469b72749
I20260812 14:22:01.351390 914 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f85fdc722e64753aac27ec3fc351894" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40751 } }
I20260812 14:22:01.351593 914 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:22:01.351670 914 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f85fdc722e64753aac27ec3fc351894, State: Initialized, Role: FOLLOWER
I20260812 14:22:01.351820 914 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f85fdc722e64753aac27ec3fc351894" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40751 } }
I20260812 14:22:01.351900 914 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:22:01.351922 914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:22:01.351986 914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:22:01.353034 914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f85fdc722e64753aac27ec3fc351894" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40751 } }
I20260812 14:22:01.353163 914 leader_election.cc:302] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f85fdc722e64753aac27ec3fc351894; no voters:
I20260812 14:22:01.353329 914 leader_election.cc:288] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:22:01.353358 919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:22:01.353564 914 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:22:01.353572 919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [term 1 LEADER]: Becoming Leader. State: Replica: 9f85fdc722e64753aac27ec3fc351894, State: Running, Role: LEADER
I20260812 14:22:01.353776 919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f85fdc722e64753aac27ec3fc351894" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40751 } }
I20260812 14:22:01.355110 922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f85fdc722e64753aac27ec3fc351894" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f85fdc722e64753aac27ec3fc351894" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40751 } } }
I20260812 14:22:01.355182 922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:01.355388 934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:22:01.355515 920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f85fdc722e64753aac27ec3fc351894. Latest consensus state: current_term: 1 leader_uuid: "9f85fdc722e64753aac27ec3fc351894" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f85fdc722e64753aac27ec3fc351894" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40751 } } }
I20260812 14:22:01.355589 920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894 [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:01.355942 934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 14:22:01 dist-test-slave-5k54 krb5kdc[816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 14:22:01.357453 934 catalog_manager.cc:1347] Generated new cluster ID: dd12a2a6701246e88e53fe04bcacad77
I20260812 14:22:01.358085 934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:22:01.362236 918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:01.350696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:43667 (local address 127.0.0.1:40751)
0812 14:22:01.350901 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:01.350911 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:01.350950 (+ 39us) server_negotiation.cc:407] Connection header received
0812 14:22:01.351343 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:01.351352 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:01.351416 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:01.351580 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:01.352353 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:01.353800 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:01.354868 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:01.355415 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:01.355562 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:01.358580 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:01.358584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:01.358584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:01.358597 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:01.360541 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:01.360968 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:01.360971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:01.360972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:01.361108 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:01.361354 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:01.361355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:01.361355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:01.361664 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:01.361827 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:01.361922 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 14:22:01.362022 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260812 14:22:01.363292 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:22:01.371764 918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:01.364709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:41431 (local address 127.0.0.1:40751)
0812 14:22:01.364744 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:01.364745 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:01.364948 (+ 203us) server_negotiation.cc:407] Connection header received
0812 14:22:01.364977 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:01.364978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:01.364990 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:01.365056 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:01.365626 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:01.366376 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:01.367315 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:01.367663 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:01.367738 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:01.368982 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:01.368983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:01.368983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:01.368990 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:01.370504 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:01.370947 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:01.370948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:01.370948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:01.371009 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:01.371239 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:01.371240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:01.371240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:01.371367 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:01.371499 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:01.371609 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 14:22:01.371670 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:22:01.381716 934 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:22:01.382195 934 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:22:01.393054 934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f85fdc722e64753aac27ec3fc351894: Generated new TSK 0
I20260812 14:22:01.393254 934 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:22:01.398186 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 836
W20260812 14:22:01.409624 814 connection.cc:535] client connection to 127.0.0.1:40751 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40751): Connection reset by peer (error 104)
2026-08-12T14:22:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:22:01 dist-test-slave-5k54 krb5kdc[950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:22:01 dist-test-slave-5k54 krb5kdc[950](info): set up 2 sockets
Aug 12 14:22:01 dist-test-slave-5k54 krb5kdc[950](info): commencing operation
krb5kdc: starting...
W20260812 14:22:02.509377 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:22:02 dist-test-slave-5k54 krb5kdc[950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:22:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:22:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:22:02.638969 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37315
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:22:02.723706 966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:22:02.723918 966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:22:02.723942 966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:22:02.725068 966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:22:02.725108 966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:22:02.725123 966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:22:02.725135 966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:22:02.725147 966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:22:02.726583 966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37315
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:22:02.726778 966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:22:02.726951 966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:22:02.731673 972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:02.731683 973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:02.731691 976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:02.732064 966 server_base.cc:1034] running on GCE node
I20260812 14:22:02.732309 966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:22:02.732594 966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:22:02.733759 966 hybrid_clock.cc:648] HybridClock initialized: now 1786544522733754 us; error 37 us; skew 500 ppm
Aug 12 14:22:02 dist-test-slave-5k54 krb5kdc[950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544522, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:22:02.740630 966 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:22:02.741107 966 webserver.cc:460] Webserver started at http://127.0.0.1:35637/ using document root <none> and password file <none>
I20260812 14:22:02.741338 966 fs_manager.cc:359] Metadata directory not provided
I20260812 14:22:02.741408 966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:22:02.741572 966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:22:02.742693 966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "adac9bf4d2d94ca7b8d39d78c1c1b25a"
format_stamp: "Formatted at 2026-08-12 14:22:02 on dist-test-slave-5k54"
server_key: "126a46163dce2b814ed5b0b0c8e513ac"
server_key_iv: "c6e1e0253450ac834ca7f9b744973af4"
server_key_version: "encryptionkey@0"
I20260812 14:22:02.743084 966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "adac9bf4d2d94ca7b8d39d78c1c1b25a"
format_stamp: "Formatted at 2026-08-12 14:22:02 on dist-test-slave-5k54"
server_key: "126a46163dce2b814ed5b0b0c8e513ac"
server_key_iv: "c6e1e0253450ac834ca7f9b744973af4"
server_key_version: "encryptionkey@0"
I20260812 14:22:02.744619 966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:22:02.745401 983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:02.745589 966 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:22:02.745695 966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "adac9bf4d2d94ca7b8d39d78c1c1b25a"
format_stamp: "Formatted at 2026-08-12 14:22:02 on dist-test-slave-5k54"
server_key: "126a46163dce2b814ed5b0b0c8e513ac"
server_key_iv: "c6e1e0253450ac834ca7f9b744973af4"
server_key_version: "encryptionkey@0"
I20260812 14:22:02.745770 966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:22:02.776253 966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:22:02.778421 966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:22:02.778596 966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:22:02.782806 966 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37315
I20260812 14:22:02.782859 1042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37315 every 8 connection(s)
I20260812 14:22:02.783237 966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:22:02.784229 1043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:22:02.784670 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 966
I20260812 14:22:02.784818 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:22:02.785027 8532 external_mini_cluster.cc:1428] Setting key 38406c3c17e401ab64ff9a9ae2cf3986
I20260812 14:22:02.787495 1043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a: Bootstrap starting.
I20260812 14:22:02.788200 1043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a: Neither blocks nor log segments found. Creating new log.
I20260812 14:22:02.788515 1043 log.cc:824] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:22:02.789439 1043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a: No bootstrap required, opened a new log
I20260812 14:22:02.792050 1043 raft_consensus.cc:348] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adac9bf4d2d94ca7b8d39d78c1c1b25a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37315 } }
I20260812 14:22:02.792228 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:02.785620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:57539 (local address 127.0.0.1:37315)
0812 14:22:02.785825 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:02.785834 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:02.785870 (+ 36us) server_negotiation.cc:407] Connection header received
0812 14:22:02.786421 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:02.786430 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:02.786495 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:02.786647 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:02.786956 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:02.788341 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:02.789730 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:02.790359 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:02.790474 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:02.791403 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:02.791409 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:02.791409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:22:02.791688 (+ 279us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:02.791736 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:22:02.791755 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:02.791846 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:02.791966 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 14:22:02.791995 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":94,"threads_started":1}
I20260812 14:22:02.792236 1043 raft_consensus.cc:374] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:22:02.792301 1043 raft_consensus.cc:729] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adac9bf4d2d94ca7b8d39d78c1c1b25a, State: Initialized, Role: FOLLOWER
I20260812 14:22:02.792421 1043 consensus_queue.cc:260] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adac9bf4d2d94ca7b8d39d78c1c1b25a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37315 } }
I20260812 14:22:02.792501 1043 raft_consensus.cc:388] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:22:02.792553 1043 raft_consensus.cc:482] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:22:02.792609 1043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:22:02.793119 1043 raft_consensus.cc:504] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adac9bf4d2d94ca7b8d39d78c1c1b25a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37315 } }
I20260812 14:22:02.793241 1043 leader_election.cc:302] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adac9bf4d2d94ca7b8d39d78c1c1b25a; no voters:
I20260812 14:22:02.793414 1043 leader_election.cc:288] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:22:02.793459 1050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:22:02.793612 1050 raft_consensus.cc:686] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [term 1 LEADER]: Becoming Leader. State: Replica: adac9bf4d2d94ca7b8d39d78c1c1b25a, State: Running, Role: LEADER
I20260812 14:22:02.793670 1043 sys_catalog.cc:564] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:22:02.793709 1050 consensus_queue.cc:237] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adac9bf4d2d94ca7b8d39d78c1c1b25a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37315 } }
I20260812 14:22:02.794124 1051 sys_catalog.cc:455] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adac9bf4d2d94ca7b8d39d78c1c1b25a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adac9bf4d2d94ca7b8d39d78c1c1b25a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37315 } } }
I20260812 14:22:02.794210 1051 sys_catalog.cc:458] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:02.794041 1050 sys_catalog.cc:455] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [sys.catalog]: SysCatalogTable state changed. Reason: New leader adac9bf4d2d94ca7b8d39d78c1c1b25a. Latest consensus state: current_term: 1 leader_uuid: "adac9bf4d2d94ca7b8d39d78c1c1b25a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adac9bf4d2d94ca7b8d39d78c1c1b25a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37315 } } }
I20260812 14:22:02.794416 1050 sys_catalog.cc:458] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:02.794533 1058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:22:02.795046 1058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:22:02.796361 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:22:02.798652 1058 catalog_manager.cc:1347] Generated new cluster ID: fe6b176616ab4ff7ac2654a9a3e78806
I20260812 14:22:02.807672 1058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:22:02.802167 1047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:02.798071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:56239 (local address 127.0.0.1:37315)
0812 14:22:02.798111 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:02.798113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:02.798244 (+ 131us) server_negotiation.cc:407] Connection header received
0812 14:22:02.798348 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:02.798349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:02.798359 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:02.798402 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:02.798628 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:02.799479 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:02.800659 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:02.800989 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:02.801077 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:02.801831 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:02.801832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:02.801833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:22:02.801837 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:02.801852 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:22:02.801857 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:02.801922 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:02.802062 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 14:22:02.802091 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:22:02.819856 1058 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:22:02.820580 1058 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:22:02.828434 1058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adac9bf4d2d94ca7b8d39d78c1c1b25a: Generated new TSK 0
I20260812 14:22:02.828689 1058 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:22:02.839551 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 966
W20260812 14:22:02.848603 948 connection.cc:535] client connection to 127.0.0.1:37315 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37315): Connection reset by peer (error 104)
2026-08-12T14:22:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:22:02 dist-test-slave-5k54 krb5kdc[1080](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:22:02 dist-test-slave-5k54 krb5kdc[1080](info): set up 2 sockets
Aug 12 14:22:02 dist-test-slave-5k54 krb5kdc[1080](info): commencing operation
krb5kdc: starting...
W20260812 14:22:03.966784 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:22:04 dist-test-slave-5k54 krb5kdc[1080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:22:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:22:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:22:04.088582 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34317
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:22:04.172835 1095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:22:04.173032 1095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:22:04.173054 1095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:22:04.174188 1095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:22:04.174283 1095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:22:04.174310 1095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:22:04.174355 1095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:22:04.174389 1095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:22:04.175927 1095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34317
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:22:04.176204 1095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:22:04.176429 1095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:22:04.181353 1095 server_base.cc:1034] running on GCE node
W20260812 14:22:04.181255 1100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:04.181280 1102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:04.181255 1104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:04.181795 1095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:22:04.182135 1095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:22:04.183334 1095 hybrid_clock.cc:648] HybridClock initialized: now 1786544524183325 us; error 41 us; skew 500 ppm
Aug 12 14:22:04 dist-test-slave-5k54 krb5kdc[1080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544524, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:22:04.185892 1095 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:22:04.186324 1095 webserver.cc:460] Webserver started at http://127.0.0.1:44347/ using document root <none> and password file <none>
I20260812 14:22:04.186575 1095 fs_manager.cc:359] Metadata directory not provided
I20260812 14:22:04.186645 1095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:22:04.186802 1095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:22:04.187917 1095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "cbc1515bb709402989722f790bad4151"
format_stamp: "Formatted at 2026-08-12 14:22:04 on dist-test-slave-5k54"
server_key: "a2918b9b2cb775a563fd6e8817071838"
server_key_iv: "aafdd9cf6fab94b480abedab00587c4a"
server_key_version: "encryptionkey@0"
I20260812 14:22:04.188297 1095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "cbc1515bb709402989722f790bad4151"
format_stamp: "Formatted at 2026-08-12 14:22:04 on dist-test-slave-5k54"
server_key: "a2918b9b2cb775a563fd6e8817071838"
server_key_iv: "aafdd9cf6fab94b480abedab00587c4a"
server_key_version: "encryptionkey@0"
I20260812 14:22:04.189621 1095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:22:04.190297 1114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:04.190486 1095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:22:04.190578 1095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "cbc1515bb709402989722f790bad4151"
format_stamp: "Formatted at 2026-08-12 14:22:04 on dist-test-slave-5k54"
server_key: "a2918b9b2cb775a563fd6e8817071838"
server_key_iv: "aafdd9cf6fab94b480abedab00587c4a"
server_key_version: "encryptionkey@0"
I20260812 14:22:04.190660 1095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:22:04.193964 1095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:22:04.195791 1095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:22:04.195942 1095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:22:04.199920 1095 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34317
I20260812 14:22:04.199962 1171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34317 every 8 connection(s)
I20260812 14:22:04.200280 1095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:22:04.201122 1172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:22:04.203357 1172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151: Bootstrap starting.
I20260812 14:22:04.203691 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 1095
I20260812 14:22:04.203824 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:22:04.204028 8532 external_mini_cluster.cc:1428] Setting key 88bba1b1069d5f8f49d744a23d2d3212
I20260812 14:22:04.204123 1172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151: Neither blocks nor log segments found. Creating new log.
I20260812 14:22:04.204483 1172 log.cc:824] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151: Log is configured to *not* fsync() on all Append() calls
I20260812 14:22:04.205221 1172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151: No bootstrap required, opened a new log
I20260812 14:22:04.206604 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:04.204518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:48663 (local address 127.0.0.1:34317)
0812 14:22:04.204737 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:04.204747 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:04.204791 (+ 44us) server_negotiation.cc:407] Connection header received
0812 14:22:04.205362 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:04.205370 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:04.205409 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:04.205525 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:04.206151 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:04.206154 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:04.206155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:22:04.206158 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:04.206196 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:22:04.206206 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:04.206280 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:04.206356 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 14:22:04.206420 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260812 14:22:04.207653 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:22:04.207867 1172 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbc1515bb709402989722f790bad4151" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34317 } }
I20260812 14:22:04.207998 1172 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:22:04.208021 1172 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbc1515bb709402989722f790bad4151, State: Initialized, Role: FOLLOWER
I20260812 14:22:04.208158 1172 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbc1515bb709402989722f790bad4151" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34317 } }
I20260812 14:22:04.208236 1172 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:22:04.208259 1172 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:22:04.208323 1172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:22:04.208992 1172 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbc1515bb709402989722f790bad4151" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34317 } }
I20260812 14:22:04.209113 1172 leader_election.cc:302] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbc1515bb709402989722f790bad4151; no voters:
I20260812 14:22:04.209308 1172 leader_election.cc:288] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:22:04.209334 1182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:22:04.209491 1182 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [term 1 LEADER]: Becoming Leader. State: Replica: cbc1515bb709402989722f790bad4151, State: Running, Role: LEADER
I20260812 14:22:04.209582 1172 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:22:04.209600 1182 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbc1515bb709402989722f790bad4151" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34317 } }
I20260812 14:22:04.210036 1184 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbc1515bb709402989722f790bad4151. Latest consensus state: current_term: 1 leader_uuid: "cbc1515bb709402989722f790bad4151" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbc1515bb709402989722f790bad4151" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34317 } } }
I20260812 14:22:04.210041 1183 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbc1515bb709402989722f790bad4151" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbc1515bb709402989722f790bad4151" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34317 } } }
I20260812 14:22:04.210160 1184 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:04.210175 1183 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151 [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:04.210685 1192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:22:04.211229 1192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:22:04.212072 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:04.209238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.29:41175 (local address 127.0.0.1:34317)
0812 14:22:04.209305 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:04.209308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:04.210977 (+ 1669us) server_negotiation.cc:407] Connection header received
0812 14:22:04.211024 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:04.211025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:04.211030 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:04.211092 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:04.211790 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:04.211792 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:04.211792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:22:04.211796 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:04.211818 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:22:04.211822 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:04.211866 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:04.212002 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 14:22:04.212014 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 14:22:04.212762 1192 catalog_manager.cc:1347] Generated new cluster ID: 2ad09c3e2b1746319bc72082cfb16e10
I20260812 14:22:04.212812 1192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:22:04.225693 1192 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:22:04.225891 1192 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:22:04.232102 1192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbc1515bb709402989722f790bad4151: Generated new TSK 0
I20260812 14:22:04.232285 1192 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:22:04.237317 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 1095
2026-08-12T14:22:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1397 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:22:04 dist-test-slave-5k54 krb5kdc[1205](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:22:04 dist-test-slave-5k54 krb5kdc[1205](info): set up 2 sockets
Aug 12 14:22:04 dist-test-slave-5k54 krb5kdc[1205](info): commencing operation
krb5kdc: starting...
W20260812 14:22:05.365026 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:22:05 dist-test-slave-5k54 krb5kdc[1205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:22:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:22:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:22:05.489961 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:32817
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:42207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:32817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:22:05.572820 1218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:22:05.573074 1218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:22:05.573113 1218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:22:05.574240 1218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:22:05.574308 1218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:22:05.574333 1218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:22:05.574363 1218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:22:05.574386 1218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:22:05.575897 1218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:42207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:32817
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:32817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:22:05.576174 1218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:22:05.576401 1218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:22:05.581352 1226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:05.581336 1224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:05.581334 1223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:05.581537 1218 server_base.cc:1034] running on GCE node
I20260812 14:22:05.581774 1218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:22:05.582095 1218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:22:05.583243 1218 hybrid_clock.cc:648] HybridClock initialized: now 1786544525583218 us; error 42 us; skew 500 ppm
Aug 12 14:22:05 dist-test-slave-5k54 krb5kdc[1205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544525, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:22:05.585848 1218 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:22:05.586268 1218 webserver.cc:460] Webserver started at http://127.8.85.62:33395/ using document root <none> and password file <none>
I20260812 14:22:05.586505 1218 fs_manager.cc:359] Metadata directory not provided
I20260812 14:22:05.586551 1218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:22:05.586721 1218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:22:05.587833 1218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "ae57612fd5d54863b5701dae39f398cc"
format_stamp: "Formatted at 2026-08-12 14:22:05 on dist-test-slave-5k54"
server_key: "9830b00a97555d7dd122e242cf92087d"
server_key_iv: "e9eb25cdda2bcdb74a25ad853ea9cce3"
server_key_version: "encryptionkey@0"
I20260812 14:22:05.588188 1218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "ae57612fd5d54863b5701dae39f398cc"
format_stamp: "Formatted at 2026-08-12 14:22:05 on dist-test-slave-5k54"
server_key: "9830b00a97555d7dd122e242cf92087d"
server_key_iv: "e9eb25cdda2bcdb74a25ad853ea9cce3"
server_key_version: "encryptionkey@0"
I20260812 14:22:05.589509 1218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:22:05.590265 1234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:05.590444 1218 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:22:05.590551 1218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "ae57612fd5d54863b5701dae39f398cc"
format_stamp: "Formatted at 2026-08-12 14:22:05 on dist-test-slave-5k54"
server_key: "9830b00a97555d7dd122e242cf92087d"
server_key_iv: "e9eb25cdda2bcdb74a25ad853ea9cce3"
server_key_version: "encryptionkey@0"
I20260812 14:22:05.590626 1218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:22:05.615283 1218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:22:05.617213 1218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:22:05.617388 1218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:22:05.621502 1218 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:32817
I20260812 14:22:05.621541 1293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:32817 every 8 connection(s)
I20260812 14:22:05.621903 1218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:22:05.622767 1294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:22:05.625006 1294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc: Bootstrap starting.
I20260812 14:22:05.625121 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 1218
I20260812 14:22:05.625288 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:22:05.625480 8532 external_mini_cluster.cc:1428] Setting key b21a9a20bd7f7757fb08c868e5b82257
I20260812 14:22:05.625700 1294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc: Neither blocks nor log segments found. Creating new log.
I20260812 14:22:05.626061 1294 log.cc:824] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc: Log is configured to *not* fsync() on all Append() calls
I20260812 14:22:05.626796 1294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc: No bootstrap required, opened a new log
I20260812 14:22:05.629123 1294 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae57612fd5d54863b5701dae39f398cc" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 32817 } }
I20260812 14:22:05.629292 1294 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:22:05.629328 1294 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae57612fd5d54863b5701dae39f398cc, State: Initialized, Role: FOLLOWER
I20260812 14:22:05.629449 1294 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae57612fd5d54863b5701dae39f398cc" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 32817 } }
I20260812 14:22:05.629519 1294 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:22:05.629547 1294 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:22:05.629590 1294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:22:05.630206 1294 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae57612fd5d54863b5701dae39f398cc" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 32817 } }
I20260812 14:22:05.630359 1294 leader_election.cc:302] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae57612fd5d54863b5701dae39f398cc; no voters:
I20260812 14:22:05.630561 1294 leader_election.cc:288] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:22:05.630595 1299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:22:05.630771 1299 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [term 1 LEADER]: Becoming Leader. State: Replica: ae57612fd5d54863b5701dae39f398cc, State: Running, Role: LEADER
I20260812 14:22:05.630923 1299 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae57612fd5d54863b5701dae39f398cc" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 32817 } }
I20260812 14:22:05.631162 1294 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:22:05.631264 1299 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae57612fd5d54863b5701dae39f398cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae57612fd5d54863b5701dae39f398cc" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 32817 } } }
I20260812 14:22:05.631332 1300 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae57612fd5d54863b5701dae39f398cc. Latest consensus state: current_term: 1 leader_uuid: "ae57612fd5d54863b5701dae39f398cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae57612fd5d54863b5701dae39f398cc" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 32817 } } }
I20260812 14:22:05.631460 1300 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:05.631696 1299 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:05.632161 1303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:22:05.632688 1303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 14:22:05 dist-test-slave-5k54 krb5kdc[1205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:22:05.634368 1303 catalog_manager.cc:1347] Generated new cluster ID: 97c79ac75a64404bb18d36d7814b97ed
I20260812 14:22:05.634421 1303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:22:05.637290 1297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:05.626067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43199 (local address 127.8.85.62:32817)
0812 14:22:05.626294 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:05.626305 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:05.626341 (+ 36us) server_negotiation.cc:407] Connection header received
0812 14:22:05.626868 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:05.626876 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:05.626960 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:05.627081 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:05.627910 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:05.628754 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:05.629868 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:05.630317 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:05.630483 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:05.633437 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:05.633441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:05.633442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:05.633456 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:05.635526 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:05.635985 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:05.635988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:05.635989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:05.636098 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:05.636339 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:05.636340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:05.636340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:05.636646 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:05.636812 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:05.636933 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 14:22:05.637054 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":77,"threads_started":1}
I20260812 14:22:05.638286 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:22:05.646909 1297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:05.639803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58125 (local address 127.8.85.62:32817)
0812 14:22:05.639837 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:05.639838 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:05.639940 (+ 102us) server_negotiation.cc:407] Connection header received
0812 14:22:05.639998 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:05.639999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:05.640007 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:05.640053 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:05.640747 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:05.641429 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:05.642471 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:05.642774 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:05.642833 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:05.644196 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:05.644197 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:05.644197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:05.644204 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:05.645707 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:05.646117 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:05.646118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:05.646118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:05.646170 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:05.646405 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:05.646406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:05.646406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:05.646526 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:05.646613 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:05.646766 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 14:22:05.646826 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:22:05.656117 1303 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:22:05.656618 1303 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:22:05.663067 1303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae57612fd5d54863b5701dae39f398cc: Generated new TSK 0
I20260812 14:22:05.663250 1303 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:22:05.668418 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 1218
W20260812 14:22:05.677426 1203 connection.cc:535] client connection to 127.8.85.62:32817 recv error: Network error: failed to read from TLS socket (remote: 127.8.85.62:32817): Connection reset by peer (error 104)
2026-08-12T14:22:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:22:05 dist-test-slave-5k54 krb5kdc[1329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:22:05 dist-test-slave-5k54 krb5kdc[1329](info): set up 2 sockets
Aug 12 14:22:05 dist-test-slave-5k54 krb5kdc[1329](info): commencing operation
krb5kdc: starting...
W20260812 14:22:06.773774 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:22:06 dist-test-slave-5k54 krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:22:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:22:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:22:06.903698 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:38427
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:34915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:38427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:22:06.986991 1344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:22:06.987197 1344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:22:06.987221 1344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:22:06.988369 1344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:22:06.988413 1344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:22:06.988435 1344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:22:06.988447 1344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:22:06.988459 1344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:22:06.989856 1344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:34915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:38427
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:38427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:22:06.990039 1344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:22:06.990209 1344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:22:06.994982 1354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:06.995029 1351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:06.995154 1344 server_base.cc:1034] running on GCE node
W20260812 14:22:06.995124 1352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:06.995579 1344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:22:06.995901 1344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:22:06.997094 1344 hybrid_clock.cc:648] HybridClock initialized: now 1786544526997077 us; error 44 us; skew 500 ppm
Aug 12 14:22:06 dist-test-slave-5k54 krb5kdc[1329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544526, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:22:06.999756 1344 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:22:07.000216 1344 webserver.cc:460] Webserver started at http://127.8.85.62:44539/ using document root <none> and password file <none>
I20260812 14:22:07.000496 1344 fs_manager.cc:359] Metadata directory not provided
I20260812 14:22:07.000574 1344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:22:07.000735 1344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:22:07.001858 1344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "182d159996c74cf2b80744b444421f21"
format_stamp: "Formatted at 2026-08-12 14:22:06 on dist-test-slave-5k54"
server_key: "023704f296ea355f02ba74f3650ed353"
server_key_iv: "e2cacda9df8de011cac87655eeb2820e"
server_key_version: "encryptionkey@0"
I20260812 14:22:07.002251 1344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "182d159996c74cf2b80744b444421f21"
format_stamp: "Formatted at 2026-08-12 14:22:06 on dist-test-slave-5k54"
server_key: "023704f296ea355f02ba74f3650ed353"
server_key_iv: "e2cacda9df8de011cac87655eeb2820e"
server_key_version: "encryptionkey@0"
I20260812 14:22:07.003768 1344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:22:07.004619 1362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:07.004810 1344 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:22:07.004920 1344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "182d159996c74cf2b80744b444421f21"
format_stamp: "Formatted at 2026-08-12 14:22:06 on dist-test-slave-5k54"
server_key: "023704f296ea355f02ba74f3650ed353"
server_key_iv: "e2cacda9df8de011cac87655eeb2820e"
server_key_version: "encryptionkey@0"
I20260812 14:22:07.005007 1344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:22:07.024350 1344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:22:07.026530 1344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:22:07.026716 1344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:22:07.031363 1344 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:38427
I20260812 14:22:07.031498 1424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:38427 every 8 connection(s)
I20260812 14:22:07.031849 1344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:22:07.032817 1425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:22:07.035267 1425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21: Bootstrap starting.
I20260812 14:22:07.035943 1425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21: Neither blocks nor log segments found. Creating new log.
I20260812 14:22:07.036207 1425 log.cc:824] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21: Log is configured to *not* fsync() on all Append() calls
I20260812 14:22:07.036923 1425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21: No bootstrap required, opened a new log
I20260812 14:22:07.038635 1425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "182d159996c74cf2b80744b444421f21" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38427 } }
I20260812 14:22:07.038785 1425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:22:07.038812 1425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182d159996c74cf2b80744b444421f21, State: Initialized, Role: FOLLOWER
I20260812 14:22:07.038877 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 1344
I20260812 14:22:07.038997 1425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "182d159996c74cf2b80744b444421f21" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38427 } }
I20260812 14:22:07.039060 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:22:07.039098 1425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:22:07.039146 1425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:22:07.039201 1425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:22:07.039285 8532 external_mini_cluster.cc:1428] Setting key 281d2ed8bcc01f7528905ed94f24f979
I20260812 14:22:07.040556 1425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "182d159996c74cf2b80744b444421f21" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38427 } }
I20260812 14:22:07.040676 1425 leader_election.cc:302] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 182d159996c74cf2b80744b444421f21; no voters:
I20260812 14:22:07.040846 1425 leader_election.cc:288] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:22:07.040881 1430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:22:07.041083 1430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [term 1 LEADER]: Becoming Leader. State: Replica: 182d159996c74cf2b80744b444421f21, State: Running, Role: LEADER
I20260812 14:22:07.041118 1425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:22:07.041199 1430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "182d159996c74cf2b80744b444421f21" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38427 } }
I20260812 14:22:07.041679 1431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "182d159996c74cf2b80744b444421f21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "182d159996c74cf2b80744b444421f21" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38427 } } }
I20260812 14:22:07.041822 1431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:07.041688 1432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 182d159996c74cf2b80744b444421f21. Latest consensus state: current_term: 1 leader_uuid: "182d159996c74cf2b80744b444421f21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "182d159996c74cf2b80744b444421f21" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 38427 } } }
I20260812 14:22:07.041888 1432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21 [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:07.042742 1442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:22:07.043648 1442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:22:07.045569 1442 catalog_manager.cc:1347] Generated new cluster ID: 622eb290877b4bd998e1bdf91848b2e5
I20260812 14:22:07.045624 1442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:22:07.046017 1429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:07.039894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40159 (local address 127.8.85.62:38427)
0812 14:22:07.040160 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:07.040174 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:07.040223 (+ 49us) server_negotiation.cc:407] Connection header received
0812 14:22:07.040354 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:07.040362 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:07.040424 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:07.040593 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:07.040881 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:07.042551 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:07.043713 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:07.044367 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:07.044501 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:07.045218 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:07.045222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:07.045223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:22:07.045437 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:07.045476 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:22:07.045491 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:07.045622 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:07.045782 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 14:22:07.045811 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":115,"threads_started":1}
I20260812 14:22:07.047101 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:22:07.052080 1429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:07.048543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56613 (local address 127.8.85.62:38427)
0812 14:22:07.048586 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:07.048588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:07.048684 (+ 96us) server_negotiation.cc:407] Connection header received
0812 14:22:07.048730 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:07.048731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:07.048751 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:07.048813 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:07.048968 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:07.049715 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:07.050718 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:07.051064 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:07.051140 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:07.051830 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:07.051832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:07.051832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:22:07.051837 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:07.051852 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:22:07.051857 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:07.051924 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:07.051996 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 14:22:07.052008 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:22:07.064190 1442 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:22:07.064698 1442 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:22:07.070859 1442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 182d159996c74cf2b80744b444421f21: Generated new TSK 0
I20260812 14:22:07.071048 1442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:22:07.075204 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 1344
W20260812 14:22:07.084649 1327 connection.cc:535] client connection to 127.8.85.62:38427 recv error: Network error: failed to read from TLS socket (remote: 127.8.85.62:38427): Connection reset by peer (error 104)
2026-08-12T14:22:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:22:07 dist-test-slave-5k54 krb5kdc[1460](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:22:07 dist-test-slave-5k54 krb5kdc[1460](info): set up 2 sockets
Aug 12 14:22:07 dist-test-slave-5k54 krb5kdc[1460](info): commencing operation
krb5kdc: starting...
W20260812 14:22:08.190400 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:22:08 dist-test-slave-5k54 krb5kdc[1460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:22:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:22:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:22:08.317487 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:35429
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:38795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:35429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:22:08.400610 1475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:22:08.400825 1475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:22:08.400856 1475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:22:08.401949 1475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:22:08.401985 1475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:22:08.402004 1475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:22:08.402017 1475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:22:08.402030 1475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:22:08.403465 1475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:38795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:35429
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:35429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:22:08.403653 1475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:22:08.403820 1475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:22:08.408689 1481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:08.408679 1480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:08.408675 1483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:08.408972 1475 server_base.cc:1034] running on GCE node
I20260812 14:22:08.409201 1475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:22:08.409442 1475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:22:08.410578 1475 hybrid_clock.cc:648] HybridClock initialized: now 1786544528410568 us; error 29 us; skew 500 ppm
Aug 12 14:22:08 dist-test-slave-5k54 krb5kdc[1460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544528, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:22:08.418584 1475 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:22:08.419103 1475 webserver.cc:460] Webserver started at http://127.8.85.62:46157/ using document root <none> and password file <none>
I20260812 14:22:08.419337 1475 fs_manager.cc:359] Metadata directory not provided
I20260812 14:22:08.419404 1475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:22:08.419593 1475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:22:08.420706 1475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "7d0131b818004b67ac649ce56a20853f"
format_stamp: "Formatted at 2026-08-12 14:22:08 on dist-test-slave-5k54"
server_key: "82b7cf0f02081e22e5c056be27cc113b"
server_key_iv: "a634f86cb3935905328133ddea9ca1db"
server_key_version: "encryptionkey@0"
I20260812 14:22:08.421092 1475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "7d0131b818004b67ac649ce56a20853f"
format_stamp: "Formatted at 2026-08-12 14:22:08 on dist-test-slave-5k54"
server_key: "82b7cf0f02081e22e5c056be27cc113b"
server_key_iv: "a634f86cb3935905328133ddea9ca1db"
server_key_version: "encryptionkey@0"
I20260812 14:22:08.422523 1475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:22:08.423303 1490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:08.423513 1475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:22:08.423627 1475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "7d0131b818004b67ac649ce56a20853f"
format_stamp: "Formatted at 2026-08-12 14:22:08 on dist-test-slave-5k54"
server_key: "82b7cf0f02081e22e5c056be27cc113b"
server_key_iv: "a634f86cb3935905328133ddea9ca1db"
server_key_version: "encryptionkey@0"
I20260812 14:22:08.423704 1475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:22:08.427068 1475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:22:08.429081 1475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:22:08.429245 1475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:22:08.433568 1475 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:35429
I20260812 14:22:08.433604 1546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:35429 every 8 connection(s)
I20260812 14:22:08.433946 1475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:22:08.434912 1547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:22:08.437589 1547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f: Bootstrap starting.
I20260812 14:22:08.438231 1547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f: Neither blocks nor log segments found. Creating new log.
I20260812 14:22:08.438544 1547 log.cc:824] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:22:08.439335 1547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f: No bootstrap required, opened a new log
I20260812 14:22:08.441064 1547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0131b818004b67ac649ce56a20853f" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35429 } }
I20260812 14:22:08.441208 1547 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:22:08.441231 1547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d0131b818004b67ac649ce56a20853f, State: Initialized, Role: FOLLOWER
I20260812 14:22:08.441434 1547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0131b818004b67ac649ce56a20853f" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35429 } }
I20260812 14:22:08.441520 1547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:22:08.441563 1547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:22:08.441613 1547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:22:08.442489 1547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0131b818004b67ac649ce56a20853f" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35429 } }
I20260812 14:22:08.442581 1547 leader_election.cc:302] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d0131b818004b67ac649ce56a20853f; no voters:
I20260812 14:22:08.442724 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 1475
I20260812 14:22:08.442800 1550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:22:08.442873 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:22:08.442831 1547 leader_election.cc:288] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:22:08.443002 1550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [term 1 LEADER]: Becoming Leader. State: Replica: 7d0131b818004b67ac649ce56a20853f, State: Running, Role: LEADER
I20260812 14:22:08.443084 1550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0131b818004b67ac649ce56a20853f" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35429 } }
I20260812 14:22:08.443122 8532 external_mini_cluster.cc:1428] Setting key a89de52528223408cfea7c940de63b11
I20260812 14:22:08.443141 1547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:22:08.443408 1552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d0131b818004b67ac649ce56a20853f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0131b818004b67ac649ce56a20853f" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35429 } } }
I20260812 14:22:08.443522 1552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:08.443688 1551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d0131b818004b67ac649ce56a20853f. Latest consensus state: current_term: 1 leader_uuid: "7d0131b818004b67ac649ce56a20853f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d0131b818004b67ac649ce56a20853f" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35429 } } }
I20260812 14:22:08.443830 1551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:08.444365 1555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:22:08.445044 1555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:22:08.446098 1558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:08.443632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41291 (local address 127.8.85.62:35429)
0812 14:22:08.443817 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:08.443829 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:08.444176 (+ 347us) server_negotiation.cc:407] Connection header received
0812 14:22:08.444333 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:08.444341 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:08.444379 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:08.444527 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:08.445638 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:08.445641 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:08.445642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:22:08.445646 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:08.445693 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:22:08.445709 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:08.445771 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:08.445871 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 14:22:08.445911 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
I20260812 14:22:08.447058 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:22:08.447794 1555 catalog_manager.cc:1347] Generated new cluster ID: 5c046651a29b4240a4580d2b0fe7d5a9
I20260812 14:22:08.447866 1555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:22:08.449838 1558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:08.448648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36053 (local address 127.8.85.62:35429)
0812 14:22:08.448682 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:08.448684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:08.448875 (+ 191us) server_negotiation.cc:407] Connection header received
0812 14:22:08.448926 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:08.448928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:08.448943 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:08.448996 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:08.449606 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:08.449608 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:08.449608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:22:08.449612 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:08.449629 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:22:08.449635 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:08.449703 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:08.449760 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 14:22:08.449779 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:22:08.460752 1555 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:22:08.460979 1555 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:22:08.462793 1555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d0131b818004b67ac649ce56a20853f: Generated new TSK 0
I20260812 14:22:08.462975 1555 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:22:08.468194 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 1475
2026-08-12T14:22:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1390 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:22:08 dist-test-slave-5k54 krb5kdc[1585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:22:08 dist-test-slave-5k54 krb5kdc[1585](info): set up 2 sockets
Aug 12 14:22:08 dist-test-slave-5k54 krb5kdc[1585](info): commencing operation
krb5kdc: starting...
W20260812 14:22:09.602044 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:22:09 dist-test-slave-5k54 krb5kdc[1585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:22:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:22:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:22:09.726933 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:35645
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:43143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:35645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:22:09.809737 1600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:22:09.809960 1600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:22:09.809985 1600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:22:09.811095 1600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:22:09.811131 1600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:22:09.811146 1600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:22:09.811158 1600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:22:09.811170 1600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:22:09.812690 1600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:43143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:35645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:35645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:22:09.812897 1600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:22:09.813076 1600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:22:09.818197 1609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:09.818166 1607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:09.818207 1600 server_base.cc:1034] running on GCE node
W20260812 14:22:09.818177 1606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:09.818558 1600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:22:09.818856 1600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:22:09.820024 1600 hybrid_clock.cc:648] HybridClock initialized: now 1786544529820028 us; error 47 us; skew 500 ppm
Aug 12 14:22:09 dist-test-slave-5k54 krb5kdc[1585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544529, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:22:09.822527 1600 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:22:09.823019 1600 webserver.cc:460] Webserver started at http://127.8.85.62:39633/ using document root <none> and password file <none>
I20260812 14:22:09.823266 1600 fs_manager.cc:359] Metadata directory not provided
I20260812 14:22:09.823336 1600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:22:09.823513 1600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:22:09.824564 1600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "570f7bce1d8b4871a2811fd0789b1050"
format_stamp: "Formatted at 2026-08-12 14:22:09 on dist-test-slave-5k54"
server_key: "95cee76a09c213e25126f47fdf6c2b22"
server_key_iv: "4bf1c945b0167b8e0757926215f72984"
server_key_version: "encryptionkey@0"
I20260812 14:22:09.824950 1600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "570f7bce1d8b4871a2811fd0789b1050"
format_stamp: "Formatted at 2026-08-12 14:22:09 on dist-test-slave-5k54"
server_key: "95cee76a09c213e25126f47fdf6c2b22"
server_key_iv: "4bf1c945b0167b8e0757926215f72984"
server_key_version: "encryptionkey@0"
I20260812 14:22:09.826323 1600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:22:09.827119 1617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:09.827347 1600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:22:09.827481 1600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "570f7bce1d8b4871a2811fd0789b1050"
format_stamp: "Formatted at 2026-08-12 14:22:09 on dist-test-slave-5k54"
server_key: "95cee76a09c213e25126f47fdf6c2b22"
server_key_iv: "4bf1c945b0167b8e0757926215f72984"
server_key_version: "encryptionkey@0"
I20260812 14:22:09.827571 1600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:22:09.838850 1600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:22:09.840834 1600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:22:09.841008 1600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:22:09.845255 1600 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:35645
I20260812 14:22:09.845295 1672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:35645 every 8 connection(s)
I20260812 14:22:09.845671 1600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:22:09.846557 1673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:22:09.848832 1673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050: Bootstrap starting.
I20260812 14:22:09.849440 1673 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050: Neither blocks nor log segments found. Creating new log.
I20260812 14:22:09.849735 1673 log.cc:824] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050: Log is configured to *not* fsync() on all Append() calls
I20260812 14:22:09.850436 1673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050: No bootstrap required, opened a new log
I20260812 14:22:09.852056 1673 raft_consensus.cc:348] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "570f7bce1d8b4871a2811fd0789b1050" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35645 } }
I20260812 14:22:09.852269 1673 raft_consensus.cc:374] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:22:09.852300 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 1600
I20260812 14:22:09.852322 1673 raft_consensus.cc:729] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 570f7bce1d8b4871a2811fd0789b1050, State: Initialized, Role: FOLLOWER
I20260812 14:22:09.852413 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:22:09.852474 1673 consensus_queue.cc:260] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "570f7bce1d8b4871a2811fd0789b1050" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35645 } }
I20260812 14:22:09.852557 1673 raft_consensus.cc:388] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:22:09.852596 1673 raft_consensus.cc:482] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:22:09.852613 8532 external_mini_cluster.cc:1428] Setting key bfe4cd4023e839c87b0cde55f5460108
I20260812 14:22:09.852648 1673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:22:09.853741 1673 raft_consensus.cc:504] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "570f7bce1d8b4871a2811fd0789b1050" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35645 } }
I20260812 14:22:09.853860 1673 leader_election.cc:302] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 570f7bce1d8b4871a2811fd0789b1050; no voters:
I20260812 14:22:09.854033 1673 leader_election.cc:288] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:22:09.854086 1679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:22:09.854244 1679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [term 1 LEADER]: Becoming Leader. State: Replica: 570f7bce1d8b4871a2811fd0789b1050, State: Running, Role: LEADER
I20260812 14:22:09.854296 1673 sys_catalog.cc:564] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:22:09.854367 1679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "570f7bce1d8b4871a2811fd0789b1050" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35645 } }
I20260812 14:22:09.854694 1679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 570f7bce1d8b4871a2811fd0789b1050. Latest consensus state: current_term: 1 leader_uuid: "570f7bce1d8b4871a2811fd0789b1050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "570f7bce1d8b4871a2811fd0789b1050" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35645 } } }
I20260812 14:22:09.854789 1679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:09.854704 1680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "570f7bce1d8b4871a2811fd0789b1050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "570f7bce1d8b4871a2811fd0789b1050" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 35645 } } }
I20260812 14:22:09.854979 1680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050 [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:09.855139 1688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:22:09.855737 1688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:22:09.857908 1688 catalog_manager.cc:1347] Generated new cluster ID: 76ad317c675342f7ae933b7adbd816b7
I20260812 14:22:09.857964 1688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:22:09 dist-test-slave-5k54 krb5kdc[1585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:22:09.864202 1678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:09.853145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48037 (local address 127.8.85.62:35645)
0812 14:22:09.853340 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:09.853349 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:09.853383 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:22:09.853484 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:09.853489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:09.853530 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:09.853638 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:09.854521 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:09.856125 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:09.857144 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:09.857637 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:09.857777 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:09.860527 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:09.860531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:09.860531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:09.860543 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:09.862419 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:09.862894 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:09.862897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:09.862897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:09.863013 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:09.863268 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:09.863269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:09.863269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:09.863580 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:09.863731 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:09.863854 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 14:22:09.863984 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":113,"threads_started":1}
I20260812 14:22:09.865375 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:22:09.873921 1678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:09.866962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37911 (local address 127.8.85.62:35645)
0812 14:22:09.866994 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:09.866996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:09.867084 (+ 88us) server_negotiation.cc:407] Connection header received
0812 14:22:09.867166 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:09.867167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:09.867185 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:09.867211 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:09.867867 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:09.868598 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:09.869690 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:09.869980 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:09.870017 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:09.871324 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:09.871325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:09.871326 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:09.871332 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:09.872822 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:09.873164 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:09.873165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:09.873165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:09.873210 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:09.873400 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:09.873400 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:09.873401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:09.873510 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:09.873582 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:09.873764 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 14:22:09.873835 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:22:09.879659 1688 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:22:09.880378 1688 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:22:09.888576 1688 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 570f7bce1d8b4871a2811fd0789b1050: Generated new TSK 0
I20260812 14:22:09.888756 1688 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:22:09.893072 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 1600
2026-08-12T14:22:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1426 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:22:09 dist-test-slave-5k54 krb5kdc[1713](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:22:09 dist-test-slave-5k54 krb5kdc[1713](info): set up 2 sockets
Aug 12 14:22:09 dist-test-slave-5k54 krb5kdc[1713](info): commencing operation
krb5kdc: starting...
W20260812 14:22:11.010407 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:22:11 dist-test-slave-5k54 krb5kdc[1713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:22:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:22:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:22:11.138706 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:40305
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:40305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:22:11.220125 1728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:22:11.220388 1728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:22:11.220427 1728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:22:11.221606 1728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:22:11.221683 1728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:22:11.221706 1728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:22:11.221740 1728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:22:11.221789 1728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:22:11.223294 1728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:40305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:40305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:22:11.223606 1728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:22:11.223850 1728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:22:11.229068 1738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:11.229095 1734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:11.229139 1728 server_base.cc:1034] running on GCE node
W20260812 14:22:11.229099 1735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:11.229517 1728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:22:11.229835 1728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:22:11.230998 1728 hybrid_clock.cc:648] HybridClock initialized: now 1786544531230973 us; error 38 us; skew 500 ppm
Aug 12 14:22:11 dist-test-slave-5k54 krb5kdc[1713](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544531, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:22:11.233778 1728 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:22:11.234287 1728 webserver.cc:460] Webserver started at http://127.8.85.62:38607/ using document root <none> and password file <none>
I20260812 14:22:11.234547 1728 fs_manager.cc:359] Metadata directory not provided
I20260812 14:22:11.234630 1728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:22:11.234833 1728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:22:11.235982 1728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "f7cddb1031844fe69a3a239e538aceb3"
format_stamp: "Formatted at 2026-08-12 14:22:11 on dist-test-slave-5k54"
server_key: "4c94c22f8b1b0616e4078e9a768ce1b5"
server_key_iv: "e08158f34cdd2fb7bf0653e374aa0932"
server_key_version: "encryptionkey@0"
I20260812 14:22:11.236373 1728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "f7cddb1031844fe69a3a239e538aceb3"
format_stamp: "Formatted at 2026-08-12 14:22:11 on dist-test-slave-5k54"
server_key: "4c94c22f8b1b0616e4078e9a768ce1b5"
server_key_iv: "e08158f34cdd2fb7bf0653e374aa0932"
server_key_version: "encryptionkey@0"
I20260812 14:22:11.237766 1728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:22:11.238539 1747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:11.238709 1728 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:22:11.238826 1728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "f7cddb1031844fe69a3a239e538aceb3"
format_stamp: "Formatted at 2026-08-12 14:22:11 on dist-test-slave-5k54"
server_key: "4c94c22f8b1b0616e4078e9a768ce1b5"
server_key_iv: "e08158f34cdd2fb7bf0653e374aa0932"
server_key_version: "encryptionkey@0"
I20260812 14:22:11.238906 1728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:22:11.252713 1728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:22:11.254796 1728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:22:11.254972 1728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:22:11.259251 1728 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:40305
I20260812 14:22:11.259289 1805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:40305 every 8 connection(s)
I20260812 14:22:11.259688 1728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:22:11.260568 1807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:22:11.262818 1807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3: Bootstrap starting.
I20260812 14:22:11.263366 1807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3: Neither blocks nor log segments found. Creating new log.
I20260812 14:22:11.263684 1807 log.cc:824] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3: Log is configured to *not* fsync() on all Append() calls
I20260812 14:22:11.263712 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 1728
I20260812 14:22:11.263860 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:22:11.264050 8532 external_mini_cluster.cc:1428] Setting key 66bee805a1312c3cce2da4b05ca6cb9f
I20260812 14:22:11.264479 1807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3: No bootstrap required, opened a new log
I20260812 14:22:11.267342 1807 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7cddb1031844fe69a3a239e538aceb3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40305 } }
I20260812 14:22:11.267531 1807 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:22:11.267563 1807 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7cddb1031844fe69a3a239e538aceb3, State: Initialized, Role: FOLLOWER
I20260812 14:22:11.267688 1807 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7cddb1031844fe69a3a239e538aceb3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40305 } }
I20260812 14:22:11.267783 1807 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:22:11.267819 1807 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:22:11.267874 1807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:22:11.268438 1807 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7cddb1031844fe69a3a239e538aceb3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40305 } }
I20260812 14:22:11.268528 1807 leader_election.cc:302] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7cddb1031844fe69a3a239e538aceb3; no voters:
I20260812 14:22:11.268703 1807 leader_election.cc:288] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:22:11.268728 1813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:22:11.268837 1813 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [term 1 LEADER]: Becoming Leader. State: Replica: f7cddb1031844fe69a3a239e538aceb3, State: Running, Role: LEADER
I20260812 14:22:11.268922 1813 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7cddb1031844fe69a3a239e538aceb3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40305 } }
I20260812 14:22:11.268981 1807 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:22:11.269352 1814 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7cddb1031844fe69a3a239e538aceb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7cddb1031844fe69a3a239e538aceb3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40305 } } }
I20260812 14:22:11.269435 1814 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:11.269639 1819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:22:11.269654 1815 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7cddb1031844fe69a3a239e538aceb3. Latest consensus state: current_term: 1 leader_uuid: "f7cddb1031844fe69a3a239e538aceb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7cddb1031844fe69a3a239e538aceb3" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 40305 } } }
I20260812 14:22:11.269845 1815 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3 [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:11.270337 1819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:22:11.271121 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:11.264597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51385 (local address 127.8.85.62:40305)
0812 14:22:11.264811 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:11.264821 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:11.264923 (+ 102us) server_negotiation.cc:407] Connection header received
0812 14:22:11.265792 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:11.265802 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:11.265867 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:11.266018 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:11.266227 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:11.267664 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:11.268664 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:11.269266 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:11.269419 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:11.270148 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:11.270153 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:11.270153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:22:11.270361 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:11.270412 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:22:11.270428 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:11.270793 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:11.270829 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 14:22:11.270903 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
I20260812 14:22:11.272143 1819 catalog_manager.cc:1347] Generated new cluster ID: 896380a2b52d4d86a2b99928c6bd6bc4
I20260812 14:22:11.272194 1819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:22:11.272235 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:22:11.277149 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:11.273753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59753 (local address 127.8.85.62:40305)
0812 14:22:11.273787 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:11.273789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:11.273920 (+ 131us) server_negotiation.cc:407] Connection header received
0812 14:22:11.273978 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:11.273979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:11.273990 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:11.274044 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:11.274170 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:11.274877 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:11.275926 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:11.276209 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:11.276265 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:11.276912 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:11.276913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:11.276914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:22:11.276918 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:11.276931 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:22:11.276935 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:11.276988 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:11.277062 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 14:22:11.277081 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:22:11.278949 1819 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:22:11.279480 1819 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:22:11.285279 1819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7cddb1031844fe69a3a239e538aceb3: Generated new TSK 0
I20260812 14:22:11.285465 1819 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:22:11.293015 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 1728
2026-08-12T14:22:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:22:11 dist-test-slave-5k54 krb5kdc[1844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:22:11 dist-test-slave-5k54 krb5kdc[1844](info): set up 2 sockets
Aug 12 14:22:11 dist-test-slave-5k54 krb5kdc[1844](info): commencing operation
krb5kdc: starting...
W20260812 14:22:12.406124 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:22:12 dist-test-slave-5k54 krb5kdc[1844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:22:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:22:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:22:12.531343 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:43551
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:40247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:43551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:22:12.612970 1860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:22:12.613197 1860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:22:12.613219 1860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:22:12.614349 1860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:22:12.614398 1860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:22:12.614411 1860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:22:12.614423 1860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:22:12.614435 1860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:22:12.615882 1860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:40247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:43551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:43551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:22:12.616130 1860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:22:12.616349 1860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:22:12.621686 1878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:12.621727 1870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:12.621683 1869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:12.621891 1860 server_base.cc:1034] running on GCE node
I20260812 14:22:12.622112 1860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:22:12.622396 1860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:22:12.623502 1860 hybrid_clock.cc:648] HybridClock initialized: now 1786544532623494 us; error 30 us; skew 500 ppm
Aug 12 14:22:12 dist-test-slave-5k54 krb5kdc[1844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544532, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:22:12.625972 1860 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:22:12.626339 1860 webserver.cc:460] Webserver started at http://127.8.85.62:38963/ using document root <none> and password file <none>
I20260812 14:22:12.626528 1860 fs_manager.cc:359] Metadata directory not provided
I20260812 14:22:12.626569 1860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:22:12.626668 1860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:22:12.627732 1860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "7adeb06554ef463f9cef0361652d3b02"
format_stamp: "Formatted at 2026-08-12 14:22:12 on dist-test-slave-5k54"
server_key: "2d6a8e90cb28aaabb8844ef8d301b724"
server_key_iv: "93cc7861588dc11b0e4d0a3f880144b8"
server_key_version: "encryptionkey@0"
I20260812 14:22:12.628077 1860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "7adeb06554ef463f9cef0361652d3b02"
format_stamp: "Formatted at 2026-08-12 14:22:12 on dist-test-slave-5k54"
server_key: "2d6a8e90cb28aaabb8844ef8d301b724"
server_key_iv: "93cc7861588dc11b0e4d0a3f880144b8"
server_key_version: "encryptionkey@0"
I20260812 14:22:12.629333 1860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:22:12.630036 1887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:12.630234 1860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:22:12.630335 1860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "7adeb06554ef463f9cef0361652d3b02"
format_stamp: "Formatted at 2026-08-12 14:22:12 on dist-test-slave-5k54"
server_key: "2d6a8e90cb28aaabb8844ef8d301b724"
server_key_iv: "93cc7861588dc11b0e4d0a3f880144b8"
server_key_version: "encryptionkey@0"
I20260812 14:22:12.630417 1860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:22:12.633767 1860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:22:12.636257 1860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:22:12.636417 1860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:22:12.640332 1860 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:43551
I20260812 14:22:12.640386 1945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:43551 every 8 connection(s)
I20260812 14:22:12.640736 1860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:22:12.641631 1946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:22:12.643901 1946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02: Bootstrap starting.
I20260812 14:22:12.644465 1946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02: Neither blocks nor log segments found. Creating new log.
I20260812 14:22:12.644760 1946 log.cc:824] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02: Log is configured to *not* fsync() on all Append() calls
I20260812 14:22:12.645458 1946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02: No bootstrap required, opened a new log
I20260812 14:22:12.646034 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 1860
I20260812 14:22:12.646184 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:22:12.646392 8532 external_mini_cluster.cc:1428] Setting key 0740a4bae102808192ae64d2f92b9d0e
I20260812 14:22:12.647555 1946 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7adeb06554ef463f9cef0361652d3b02" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43551 } }
I20260812 14:22:12.647735 1946 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:22:12.647778 1946 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7adeb06554ef463f9cef0361652d3b02, State: Initialized, Role: FOLLOWER
I20260812 14:22:12.647934 1946 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7adeb06554ef463f9cef0361652d3b02" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43551 } }
I20260812 14:22:12.648020 1946 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:22:12.648061 1946 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:22:12.648115 1946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:22:12.648795 1951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:12.646955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32999 (local address 127.8.85.62:43551)
0812 14:22:12.647161 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:12.647168 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:12.647193 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:22:12.647514 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:12.647520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:12.647544 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:12.647655 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:12.648335 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:12.648338 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:12.648339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:22:12.648342 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:12.648388 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:22:12.648399 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:12.648453 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:12.648548 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 14:22:12.648580 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260812 14:22:12.649187 1946 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7adeb06554ef463f9cef0361652d3b02" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43551 } }
I20260812 14:22:12.649309 1946 leader_election.cc:302] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7adeb06554ef463f9cef0361652d3b02; no voters:
I20260812 14:22:12.649487 1946 leader_election.cc:288] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:22:12.649504 1952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:22:12.649616 1952 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [term 1 LEADER]: Becoming Leader. State: Replica: 7adeb06554ef463f9cef0361652d3b02, State: Running, Role: LEADER
I20260812 14:22:12.649698 1952 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7adeb06554ef463f9cef0361652d3b02" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43551 } }
I20260812 14:22:12.649756 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:22:12.649729 1946 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:22:12.650043 1954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7adeb06554ef463f9cef0361652d3b02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7adeb06554ef463f9cef0361652d3b02" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43551 } } }
I20260812 14:22:12.650134 1954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:12.650313 1953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7adeb06554ef463f9cef0361652d3b02. Latest consensus state: current_term: 1 leader_uuid: "7adeb06554ef463f9cef0361652d3b02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7adeb06554ef463f9cef0361652d3b02" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43551 } } }
I20260812 14:22:12.650372 1953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02 [sys.catalog]: This master's current role is: LEADER
W20260812 14:22:12.651629 1974 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:22:12.651726 1974 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:22:12.651822 1957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:22:12.652029 1957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:22:12.652616 1951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:12.651326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56315 (local address 127.8.85.62:43551)
0812 14:22:12.651364 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:12.651367 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:12.651633 (+ 266us) server_negotiation.cc:407] Connection header received
0812 14:22:12.651723 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:12.651724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:12.651739 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:12.651765 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:12.652398 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:12.652399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:12.652400 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:22:12.652404 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:12.652416 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:22:12.652420 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:12.652448 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:12.652546 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 14:22:12.652558 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:22:12.653378 1957 catalog_manager.cc:1347] Generated new cluster ID: 38925bf695c646a58c9e5c197d0f4f26
I20260812 14:22:12.653429 1957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:22:12.684139 1957 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:22:12.684355 1957 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:22:12.690527 1957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7adeb06554ef463f9cef0361652d3b02: Generated new TSK 0
I20260812 14:22:12.690724 1957 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:22:12.693195 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 1860
2026-08-12T14:22:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1400 ms)
[----------] 18 tests from AuthTokenIssuingTest (25448 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:22:12 dist-test-slave-5k54 krb5kdc[1982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:22:12 dist-test-slave-5k54 krb5kdc[1982](info): set up 2 sockets
Aug 12 14:22:12 dist-test-slave-5k54 krb5kdc[1982](info): commencing operation
krb5kdc: starting...
W20260812 14:22:13.817879 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:22:13 dist-test-slave-5k54 krb5kdc[1982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:22:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:22:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:22:13.946564 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:45129
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:45129,127.8.85.61:34303,127.8.85.60:43325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:22:14.029677 1996 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:22:14.029889 1996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:22:14.029907 1996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:22:14.029927 1996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:22:14.031070 1996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:22:14.031116 1996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:22:14.031131 1996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:22:14.031143 1996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:22:14.031155 1996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:22:14.032713 1996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:45129,127.8.85.61:34303,127.8.85.60:43325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:45129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:22:14.032970 1996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:22:14.033213 1996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:22:14.038172 2004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:14.038314 1996 server_base.cc:1034] running on GCE node
W20260812 14:22:14.038226 2002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:14.038183 2001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:14.038692 1996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:22:14.038977 1996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:22:14.040144 1996 hybrid_clock.cc:648] HybridClock initialized: now 1786544534040131 us; error 42 us; skew 500 ppm
Aug 12 14:22:14 dist-test-slave-5k54 krb5kdc[1982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:22:14.049947 1996 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:22:14.050403 1996 webserver.cc:460] Webserver started at http://127.8.85.62:33559/ using document root <none> and password file <none>
I20260812 14:22:14.050655 1996 fs_manager.cc:359] Metadata directory not provided
I20260812 14:22:14.050726 1996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:22:14.050884 1996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:22:14.052057 1996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "b6648a2023f04a1b925af4b9221bd80a"
format_stamp: "Formatted at 2026-08-12 14:22:14 on dist-test-slave-5k54"
server_key: "be690ffd091daf54a4f22e5a84afea93"
server_key_iv: "3dc023526a3850617bf7d754ae937d35"
server_key_version: "encryptionkey@0"
I20260812 14:22:14.052433 1996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "b6648a2023f04a1b925af4b9221bd80a"
format_stamp: "Formatted at 2026-08-12 14:22:14 on dist-test-slave-5k54"
server_key: "be690ffd091daf54a4f22e5a84afea93"
server_key_iv: "3dc023526a3850617bf7d754ae937d35"
server_key_version: "encryptionkey@0"
I20260812 14:22:14.053794 1996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:22:14.054653 2011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:14.054852 1996 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:22:14.054971 1996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "b6648a2023f04a1b925af4b9221bd80a"
format_stamp: "Formatted at 2026-08-12 14:22:14 on dist-test-slave-5k54"
server_key: "be690ffd091daf54a4f22e5a84afea93"
server_key_iv: "3dc023526a3850617bf7d754ae937d35"
server_key_version: "encryptionkey@0"
I20260812 14:22:14.055047 1996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:22:14.073879 1996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:22:14.075836 1996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:22:14.076011 1996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:22:14.080107 1996 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:45129
I20260812 14:22:14.080156 2070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:45129 every 8 connection(s)
I20260812 14:22:14.080541 1996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:22:14.081318 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 1996
I20260812 14:22:14.081444 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:22:14.081666 8532 external_mini_cluster.cc:1428] Setting key 944325d72337857e8ed80470ae85c0b9
I20260812 14:22:14.081748 2071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:22:14.084108 2071 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.85.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.61@KRBTEST.COM" created.
Aug 12 14:22:14 dist-test-slave-5k54 krb5kdc[1982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:22:14.097407 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.084863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45129 (local address 127.0.0.1:48756)
0812 14:22:14.085031 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.085068 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:14.085595 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:14.085889 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:14.085891 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:14.086253 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:14.086417 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.086423 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.087468 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.087469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.088147 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.088149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.088711 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.088712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.088779 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.089316 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:14.089324 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:14.093973 (+ 4649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:14.095922 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.095929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.095931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.096128 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.096412 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.096413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.096413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.096499 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.096871 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:14.096876 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:14.096947 (+ 71us) client_negotiation.cc:769] Sending connection context
0812 14:22:14.097052 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 14:22:14.097222 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260812 14:22:14.097506 2076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.085063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48756 (local address 127.8.85.62:45129)
0812 14:22:14.085194 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.085199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.085233 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:22:14.085758 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.085761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.085786 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.085884 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.086520 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.087342 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.088251 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.088602 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.088707 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.094111 (+ 5404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:14.094119 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:14.094119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:14.094126 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:14.095843 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.096244 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.096247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.096248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.096340 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.096567 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.096568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.096569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.096792 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:14.096882 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:14.097123 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 14:22:14.097340 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
I20260812 14:22:14.098302 2071 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:22:14.098706 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.098564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:34303 (local address 127.0.0.1:48234)
0812 14:22:14.098636 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.098685 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:34303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 14:22:14.098869 2012 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.85.61:34303: connect: Connection refused (error 111)
I20260812 14:22:14.099327 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.099229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:34303 (local address 127.0.0.1:48246)
0812 14:22:14.099278 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.099309 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:34303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 14:22:14.099457 2071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.61:34303: Network error: Client connection negotiation failed: client connection to 127.8.85.61:34303: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.61.keytab.
Entry for principal kudu/127.8.85.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.61.keytab.
I20260812 14:22:14.110426 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.61:34303
--webserver_interface=127.8.85.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.61.keytab
--principal=kudu/127.8.85.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:45129,127.8.85.61:34303,127.8.85.60:43325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:22:14.148705 2071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } attempt: 1
I20260812 14:22:14.149220 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.149059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:34303 (local address 127.0.0.1:48252)
0812 14:22:14.149128 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.149191 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:34303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 14:22:14.149593 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.149495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:34303 (local address 127.0.0.1:48266)
0812 14:22:14.149541 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.149569 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:34303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 14:22:14.149685 2071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.61:34303: Network error: Client connection negotiation failed: client connection to 127.8.85.61:34303: connect: Connection refused (error 111)
W20260812 14:22:14.192618 2079 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:22:14.192817 2079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:22:14.192834 2079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:22:14.192853 2079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:22:14.193984 2079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:22:14.194022 2079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:22:14.194036 2079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:22:14.194048 2079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:22:14.194099 2079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:22:14.195545 2079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:45129,127.8.85.61:34303,127.8.85.60:43325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.61.keytab
--principal=kudu/127.8.85.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.61:34303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.85.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:22:14.195789 2079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:22:14.196014 2079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:22:14.200743 2086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:14.200752 2090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:14.201025 2079 server_base.cc:1034] running on GCE node
W20260812 14:22:14.200752 2085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:14.201308 2079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:22:14.201610 2079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:22:14.202782 2079 hybrid_clock.cc:648] HybridClock initialized: now 1786544534202770 us; error 28 us; skew 500 ppm
Aug 12 14:22:14 dist-test-slave-5k54 krb5kdc[1982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:22:14.205556 2079 init.cc:377] Logged in from keytab as kudu/127.8.85.61@KRBTEST.COM (short username kudu)
I20260812 14:22:14.206074 2079 webserver.cc:460] Webserver started at http://127.8.85.61:44525/ using document root <none> and password file <none>
I20260812 14:22:14.206305 2079 fs_manager.cc:359] Metadata directory not provided
I20260812 14:22:14.206351 2079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:22:14.206518 2079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:22:14.207652 2079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/data/instance:
uuid: "f4825d7fd37740448b4f90eaab1cf065"
format_stamp: "Formatted at 2026-08-12 14:22:14 on dist-test-slave-5k54"
server_key: "b7c473a1c222d7a11a0bee0bd807ff7b"
server_key_iv: "20b7096045f0ca97e7636368c138483e"
server_key_version: "encryptionkey@0"
I20260812 14:22:14.208024 2079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/wal/instance:
uuid: "f4825d7fd37740448b4f90eaab1cf065"
format_stamp: "Formatted at 2026-08-12 14:22:14 on dist-test-slave-5k54"
server_key: "b7c473a1c222d7a11a0bee0bd807ff7b"
server_key_iv: "20b7096045f0ca97e7636368c138483e"
server_key_version: "encryptionkey@0"
I20260812 14:22:14.209367 2079 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 14:22:14.210156 2097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:14.210354 2079 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:22:14.210448 2079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/wal
uuid: "f4825d7fd37740448b4f90eaab1cf065"
format_stamp: "Formatted at 2026-08-12 14:22:14 on dist-test-slave-5k54"
server_key: "b7c473a1c222d7a11a0bee0bd807ff7b"
server_key_iv: "20b7096045f0ca97e7636368c138483e"
server_key_version: "encryptionkey@0"
I20260812 14:22:14.210503 2079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:22:14.217502 2079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:22:14.217855 2071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } attempt: 2
I20260812 14:22:14.218247 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.218119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:34303 (local address 127.0.0.1:48270)
0812 14:22:14.218179 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.218220 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:34303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 14:22:14.218572 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.218470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:34303 (local address 127.0.0.1:48284)
0812 14:22:14.218517 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.218542 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:34303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 14:22:14.218677 2071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.61:34303: Network error: Client connection negotiation failed: client connection to 127.8.85.61:34303: connect: Connection refused (error 111)
I20260812 14:22:14.219753 2079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:22:14.219910 2079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:22:14.223865 2079 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.61:34303
I20260812 14:22:14.223903 2153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.61:34303 every 8 connection(s)
I20260812 14:22:14.224241 2079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/data/info.pb
I20260812 14:22:14.225253 2154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:22:14.225975 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 2079
I20260812 14:22:14.226060 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-1/wal/instance
I20260812 14:22:14.226269 8532 external_mini_cluster.cc:1428] Setting key 9dee598be808fd8b3021c421f22dd551
I20260812 14:22:14.226950 2154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:22:14 dist-test-slave-5k54 krb5kdc[1982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.61@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.85.60@KRBTEST.COM; defaulting to no policy
I20260812 14:22:14.236848 2076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.227707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48770 (local address 127.8.85.62:45129)
0812 14:22:14.227749 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.227752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.228024 (+ 272us) server_negotiation.cc:407] Connection header received
0812 14:22:14.228734 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.228735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.228749 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.228802 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.229658 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.230146 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.231679 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.231888 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.231989 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.234167 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:14.234168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:14.234169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:14.234176 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:14.235739 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.236107 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.236108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.236109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.236163 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.236365 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.236366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.236366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.236459 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:14.236502 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:14.236721 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 14:22:14.236785 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:22:14.237051 2158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.227687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45129 (local address 127.0.0.1:48770)
0812 14:22:14.227859 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.227908 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:14.228622 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:14.228877 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:14.228890 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:14.229259 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:14.229562 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.229568 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.230248 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.230249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.231598 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.231601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.232033 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.232035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.232165 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.232665 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:14.232683 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:14.234067 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:14.235813 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.235820 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.235821 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.236046 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.236224 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.236224 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.236225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.236319 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.236507 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:14.236512 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:14.236620 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 14:22:14.236699 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 14:22:14.236813 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.60@KRBTEST.COM" created.
I20260812 14:22:14.237654 2154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:22:14 dist-test-slave-5k54 krb5kdc[1982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.61@KRBTEST.COM for kudu/127.8.85.61@KRBTEST.COM
I20260812 14:22:14.245342 2158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.237910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:34303 (local address 127.0.0.1:48300)
0812 14:22:14.237963 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.237974 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:14.238022 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:14.238273 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:14.238274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:14.238432 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:14.238491 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.238493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.239343 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.239345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.239976 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.239978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.240617 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.240618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.240672 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.241061 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:14.241068 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:14.242163 (+ 1095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:14.244267 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.244268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.244268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.244437 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.244653 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.244654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.244654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.244709 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.245013 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:14.245014 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:14.245051 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 14:22:14.245121 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 14:22:14.245216 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 14:22:14.245414 2160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.237979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48300 (local address 127.8.85.61:34303)
0812 14:22:14.238123 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.238128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.238146 (+ 18us) server_negotiation.cc:407] Connection header received
0812 14:22:14.238175 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.238176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.238188 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.238248 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.238565 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.239240 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.240056 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.240538 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.240606 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.242246 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:14.242254 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:14.242255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:14.242261 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:14.244169 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.244528 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.244531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.244531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.244586 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.244792 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.244793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.244793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.244935 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:14.245023 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:14.245151 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 14:22:14.245268 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260812 14:22:14.245949 2154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:22:14.246388 2158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.246275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:43325 (local address 127.0.0.1:33540)
0812 14:22:14.246315 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.246357 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260812 14:22:14.246577 2101 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
I20260812 14:22:14.246925 2158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.246869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:43325 (local address 127.0.0.1:33552)
0812 14:22:14.246890 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.246911 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260812 14:22:14.247035 2154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.60:43325: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.60.keytab.
Entry for principal kudu/127.8.85.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.60.keytab.
I20260812 14:22:14.255174 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.60:43325
--webserver_interface=127.8.85.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.60.keytab
--principal=kudu/127.8.85.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:45129,127.8.85.61:34303,127.8.85.60:43325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:22:14.296269 2154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } attempt: 1
I20260812 14:22:14.296746 2158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.296616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:43325 (local address 127.0.0.1:33560)
0812 14:22:14.296680 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.296715 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 14:22:14.297191 2158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.297099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:43325 (local address 127.0.0.1:33566)
0812 14:22:14.297146 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.297172 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 14:22:14.297300 2154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.60:43325: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
I20260812 14:22:14.309818 2071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } attempt: 3
Aug 12 14:22:14 dist-test-slave-5k54 krb5kdc[1982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for kudu/127.8.85.61@KRBTEST.COM
I20260812 14:22:14.317071 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.310159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:34303 (local address 127.0.0.1:36772)
0812 14:22:14.310219 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.310231 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:14.310287 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:14.310458 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:14.310458 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:14.310617 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:14.310682 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.310684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.311383 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.311384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.311968 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.311970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.312348 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.312348 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.312435 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.312940 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:14.312945 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:14.314313 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:14.316128 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.316129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.316130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.316298 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.316506 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.316506 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.316507 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.316558 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.316816 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:14.316816 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:14.316856 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 14:22:14.316932 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 14:22:14.317007 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 14:22:14.317152 2160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.310216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36772 (local address 127.8.85.61:34303)
0812 14:22:14.310274 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.310277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.310289 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:22:14.310356 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.310357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.310370 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.310437 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.310763 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.311279 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.312053 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.312271 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.312351 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.314405 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:14.314406 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:14.314407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:14.314415 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:14.316043 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.316370 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.316371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.316372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.316440 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.316616 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.316617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.316617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.316752 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:14.316819 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:14.316978 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 14:22:14.317061 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:22:14.317492 2071 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:22:14.317812 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.317718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:43325 (local address 127.0.0.1:51758)
0812 14:22:14.317760 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.317791 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 14:22:14.318202 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.318088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:43325 (local address 127.0.0.1:51762)
0812 14:22:14.318137 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.318175 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 14:22:14.318300 2071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.60:43325: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
W20260812 14:22:14.340175 2163 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:22:14.340373 2163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:22:14.340391 2163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:22:14.340412 2163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:22:14.341544 2163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:22:14.341585 2163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:22:14.341600 2163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:22:14.341612 2163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:22:14.341662 2163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:22:14.343127 2163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:45129,127.8.85.61:34303,127.8.85.60:43325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.60.keytab
--principal=kudu/127.8.85.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.60:43325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.85.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:22:14.343374 2163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:22:14.343623 2163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:22:14.348385 2170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:14.348392 2172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:14.348604 2163 server_base.cc:1034] running on GCE node
W20260812 14:22:14.348394 2169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:14.348932 2163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:22:14.349180 2163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:22:14.349462 2071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } attempt: 1
I20260812 14:22:14.349838 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.349724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:43325 (local address 127.0.0.1:51770)
0812 14:22:14.349782 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.349812 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 14:22:14.350147 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.350066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:43325 (local address 127.0.0.1:51776)
0812 14:22:14.350107 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.350131 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 14:22:14.350229 2071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.60:43325: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
I20260812 14:22:14.350317 2163 hybrid_clock.cc:648] HybridClock initialized: now 1786544534350291 us; error 37 us; skew 500 ppm
Aug 12 14:22:14 dist-test-slave-5k54 krb5kdc[1982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:22:14.352946 2163 init.cc:377] Logged in from keytab as kudu/127.8.85.60@KRBTEST.COM (short username kudu)
I20260812 14:22:14.353511 2163 webserver.cc:460] Webserver started at http://127.8.85.60:33263/ using document root <none> and password file <none>
I20260812 14:22:14.353889 2163 fs_manager.cc:359] Metadata directory not provided
I20260812 14:22:14.353981 2163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:22:14.354140 2163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:22:14.355165 2163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/data/instance:
uuid: "912d8da8a65c46cd8ed3399f4b6b18f9"
format_stamp: "Formatted at 2026-08-12 14:22:14 on dist-test-slave-5k54"
server_key: "d95a62ade79fa7f7ec0280e45ebedec7"
server_key_iv: "3455a4764a2e7108701c8c23eb7bf30f"
server_key_version: "encryptionkey@0"
I20260812 14:22:14.355604 2163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/wal/instance:
uuid: "912d8da8a65c46cd8ed3399f4b6b18f9"
format_stamp: "Formatted at 2026-08-12 14:22:14 on dist-test-slave-5k54"
server_key: "d95a62ade79fa7f7ec0280e45ebedec7"
server_key_iv: "3455a4764a2e7108701c8c23eb7bf30f"
server_key_version: "encryptionkey@0"
I20260812 14:22:14.357070 2163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:22:14.357992 2188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:14.358191 2163 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:22:14.358295 2163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/wal
uuid: "912d8da8a65c46cd8ed3399f4b6b18f9"
format_stamp: "Formatted at 2026-08-12 14:22:14 on dist-test-slave-5k54"
server_key: "d95a62ade79fa7f7ec0280e45ebedec7"
server_key_iv: "3455a4764a2e7108701c8c23eb7bf30f"
server_key_version: "encryptionkey@0"
I20260812 14:22:14.358383 2163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:22:14.365447 2154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } attempt: 2
I20260812 14:22:14.365859 2158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.365717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:43325 (local address 127.0.0.1:51790)
0812 14:22:14.365783 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.365824 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 14:22:14.366202 2158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.366112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:43325 (local address 127.0.0.1:51792)
0812 14:22:14.366155 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.366183 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 14:22:14.366325 2154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.60:43325: Network error: Client connection negotiation failed: client connection to 127.8.85.60:43325: connect: Connection refused (error 111)
I20260812 14:22:14.368592 2163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:22:14.370839 2163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:22:14.370999 2163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:22:14.375063 2163 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.60:43325
I20260812 14:22:14.375094 2248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.60:43325 every 8 connection(s)
I20260812 14:22:14.375577 2163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/data/info.pb
I20260812 14:22:14.376576 2249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:22:14.378062 2249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:22:14.380714 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 2163
I20260812 14:22:14.380810 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786544263511571-8532-0/minicluster-data/master-2/wal/instance
I20260812 14:22:14.381054 8532 external_mini_cluster.cc:1428] Setting key f3704887cdb58dddc628aace7494f4ed
Aug 12 14:22:14 dist-test-slave-5k54 krb5kdc[1982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.60@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
Aug 12 14:22:14 dist-test-slave-5k54 krb5kdc[1982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.60@KRBTEST.COM
I20260812 14:22:14.388731 2076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.378928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33378 (local address 127.8.85.62:45129)
0812 14:22:14.379004 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.379005 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.379213 (+ 208us) server_negotiation.cc:407] Connection header received
0812 14:22:14.379897 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.379898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.379909 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.379978 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.380614 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.381394 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.382869 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.383067 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.383146 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.385515 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:14.385516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:14.385516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:14.385523 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:14.387170 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.387744 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.387745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.387745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.387796 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.388103 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.388104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.388105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.388212 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:14.388261 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:14.388606 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 14:22:14.388668 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 14:22:14.389039 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.378915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:45129 (local address 127.0.0.1:33378)
0812 14:22:14.379069 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.379109 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:14.379655 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:14.380079 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:14.380082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:14.380321 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:14.380508 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.380511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.381518 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.381519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.382782 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.382785 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.383176 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.383177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.383318 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.383826 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:14.383835 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:14.385405 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:14.387276 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.387280 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.387282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.387670 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.387871 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.387872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.387873 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.388014 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.388278 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:14.388283 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:14.388432 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 14:22:14.388559 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 14:22:14.388756 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260812 14:22:14.389643 2249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:22:14.390497 2253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.381456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51794 (local address 127.8.85.60:43325)
0812 14:22:14.381602 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.381607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.381662 (+ 55us) server_negotiation.cc:407] Connection header received
0812 14:22:14.381750 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.381752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.381768 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.381829 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.382570 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.383254 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.384281 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.384566 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.384666 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.387010 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:14.387018 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:14.387018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:14.387024 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:14.389100 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.389534 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.389537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.389537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.389601 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.389839 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.389841 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.389841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.390004 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:14.390145 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:14.390248 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 14:22:14.390398 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":91,"threads_started":1}
Aug 12 14:22:14 dist-test-slave-5k54 krb5kdc[1982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.60@KRBTEST.COM for kudu/127.8.85.61@KRBTEST.COM
Aug 12 14:22:14 dist-test-slave-5k54 krb5kdc[1982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:22:14.396849 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.389898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:34303 (local address 127.0.0.1:36786)
0812 14:22:14.390014 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.390025 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:14.390072 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:14.390201 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:14.390202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:14.390371 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:14.390443 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.390445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.391312 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.391314 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.391915 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.391917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.392270 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.392271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.392338 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.392843 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:14.392848 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:14.394073 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:14.395809 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.395810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.395810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.395985 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.396205 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.396206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.396206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.396270 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.396528 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:14.396529 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:14.396570 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 14:22:14.396671 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 14:22:14.396761 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260812 14:22:14.396857 2160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.389972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36786 (local address 127.8.85.61:34303)
0812 14:22:14.390014 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.390016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.390058 (+ 42us) server_negotiation.cc:407] Connection header received
0812 14:22:14.390133 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.390134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.390142 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.390197 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.390536 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.391206 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.392002 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.392201 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.392252 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.394168 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:14.394169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:14.394170 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:14.394178 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:14.395726 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.396072 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.396074 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.396074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.396133 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.396332 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.396333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.396334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.396458 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:14.396534 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:14.396699 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 14:22:14.396772 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:22:14.397375 2249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:22:14.398607 2076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.391266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33392 (local address 127.8.85.62:45129)
0812 14:22:14.391303 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.391305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.391312 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:22:14.391336 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.391336 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.391345 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.391411 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.391872 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.392449 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.393094 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.393280 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.393353 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.395799 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:14.395809 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:14.395810 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:14.395816 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:14.397405 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.397800 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.397801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.397801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.397848 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.398050 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.398050 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.398051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.398159 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:14.398393 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:14.398432 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 14:22:14.398514 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 12 14:22:14 dist-test-slave-5k54 krb5kdc[1982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.60@KRBTEST.COM for kudu/127.8.85.60@KRBTEST.COM
Aug 12 14:22:14 dist-test-slave-5k54 krb5kdc[1982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.61@KRBTEST.COM
I20260812 14:22:14.404493 2251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.397622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:43325 (local address 127.0.0.1:51796)
0812 14:22:14.397661 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.397670 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:14.397706 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:14.397837 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:14.397838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:14.397983 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:14.398017 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.398019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.399095 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.399096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.399716 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.399719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.400068 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.400069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.400116 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.400527 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:14.400532 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:14.401941 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:14.403524 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.403526 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.403526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.403734 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.403952 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.403953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.403953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.404018 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.404249 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:14.404250 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:14.404295 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 14:22:14.404375 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 14:22:14.404439 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 14:22:14.404614 2253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.397675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51796 (local address 127.8.85.60:43325)
0812 14:22:14.397713 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.397715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.397723 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:22:14.397744 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.397745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.397755 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.397805 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.398088 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.399025 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.399793 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.399998 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.400070 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.402003 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:14.402004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:14.402004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:14.402011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:14.403370 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.403808 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.403809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.403809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.403881 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.404088 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.404088 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.404089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.404180 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:14.404230 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:14.404417 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 14:22:14.404493 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:22:14.406613 2249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9: Bootstrap starting.
I20260812 14:22:14.407161 2160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.399207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36790 (local address 127.8.85.61:34303)
0812 14:22:14.399244 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.399246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.399253 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:22:14.399294 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.399295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.399303 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.399363 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.399857 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.400373 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.401321 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.401681 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.401736 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.404328 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:14.404329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:14.404329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:14.404334 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:14.406038 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.406387 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.406388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.406388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.406447 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.406651 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.406652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.406652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.406769 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:14.406932 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:14.406985 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 14:22:14.407057 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:22:14.407650 2249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9: Neither blocks nor log segments found. Creating new log.
I20260812 14:22:14.408001 2249 log.cc:824] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9: Log is configured to *not* fsync() on all Append() calls
I20260812 14:22:14.408697 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:22:14.408937 2249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9: No bootstrap required, opened a new log
I20260812 14:22:14.409822 2041 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b6648a2023f04a1b925af4b9221bd80a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33392
I20260812 14:22:14.410440 2249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } }
I20260812 14:22:14.410600 2249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 912d8da8a65c46cd8ed3399f4b6b18f9, State: Initialized, Role: FOLLOWER
I20260812 14:22:14.410686 2249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } }
I20260812 14:22:14.410991 2249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:22:14.410981 2255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } } }
I20260812 14:22:14.411091 2255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:22:14.412911 2268 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:22:14.412971 2268 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:22:14.413226 2163 master_runner.cc:186] Error getting master registration for 127.8.85.62:45129: OK, instance_id { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" instance_seqno: 1786544534078923 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:22:14.416141 2163 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:22:14.425372 2071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } attempt: 2
Aug 12 14:22:14 dist-test-slave-5k54 krb5kdc[1982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for kudu/127.8.85.60@KRBTEST.COM
I20260812 14:22:14.432477 2075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.425673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:43325 (local address 127.0.0.1:51804)
0812 14:22:14.425724 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.425736 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:14.425779 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:14.425927 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:14.425928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:14.426096 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:14.426136 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.426138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.426762 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.426763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.427262 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.427263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.427683 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.427684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.427742 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.428218 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:14.428224 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:14.429558 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:14.431369 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.431370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.431370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.431592 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.431801 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.431802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.431803 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.431863 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.432148 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:14.432149 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:14.432193 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 14:22:14.432281 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 14:22:14.432388 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 14:22:14.432476 2253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.425711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51804 (local address 127.8.85.60:43325)
0812 14:22:14.425749 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.425751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.425762 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:22:14.425831 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.425832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.425841 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.425921 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.426215 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.426679 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.427344 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.427601 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.427674 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.429647 (+ 1973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:14.429648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:14.429649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:14.429657 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:14.431287 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.431667 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.431668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.431669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.431725 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.431936 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.431937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.431937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.432076 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:14.432156 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:14.432307 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 14:22:14.432382 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:22:14.433719 2071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a: Bootstrap starting.
I20260812 14:22:14.434475 2071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a: Neither blocks nor log segments found. Creating new log.
I20260812 14:22:14.434801 2071 log.cc:824] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:22:14.435653 2071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a: No bootstrap required, opened a new log
I20260812 14:22:14.436673 2071 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } }
I20260812 14:22:14.436779 2071 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6648a2023f04a1b925af4b9221bd80a, State: Initialized, Role: FOLLOWER
I20260812 14:22:14.436918 2071 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } }
I20260812 14:22:14.437191 2271 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } } }
I20260812 14:22:14.437252 2071 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:22:14.437268 2271 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:22:14.437392 2041 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 14:22:14.437474 2041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:22:14.438364 2041 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } }
W20260812 14:22:14.439064 2282 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:22:14.439119 2282 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:22:14.439210 2041 leader_election.cc:288] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [CANDIDATE]: Term 1 election: Requested vote from peers f4825d7fd37740448b4f90eaab1cf065 (127.8.85.61:34303), 912d8da8a65c46cd8ed3399f4b6b18f9 (127.8.85.60:43325)
I20260812 14:22:14.439503 2219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6648a2023f04a1b925af4b9221bd80a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9"
I20260812 14:22:14.439651 2219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:22:14.441035 2219 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6648a2023f04a1b925af4b9221bd80a in term 1.
I20260812 14:22:14.441325 2015 leader_election.cc:302] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 912d8da8a65c46cd8ed3399f4b6b18f9, b6648a2023f04a1b925af4b9221bd80a; no voters:
I20260812 14:22:14.441453 2271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:22:14.441587 2271 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [term 1 LEADER]: Becoming Leader. State: Replica: b6648a2023f04a1b925af4b9221bd80a, State: Running, Role: LEADER
I20260812 14:22:14.441674 2271 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } }
I20260812 14:22:14.442101 2271 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6648a2023f04a1b925af4b9221bd80a. Latest consensus state: current_term: 1 leader_uuid: "b6648a2023f04a1b925af4b9221bd80a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } } }
I20260812 14:22:14.442188 2271 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:14.439917 1996 master_runner.cc:186] Error getting master registration for 127.8.85.61:34303: OK, instance_id { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" instance_seqno: 1786544534222651 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:22:14.442600 1996 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:22:14.443346 2286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:22:14.443583 2286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:22:14.445672 2219 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9 [term 1 FOLLOWER]: Refusing update from remote peer b6648a2023f04a1b925af4b9221bd80a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:22:14.445981 2271 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [LEADER]: Connected to new peer: Peer: permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:22:14.447160 2255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6648a2023f04a1b925af4b9221bd80a. Latest consensus state: current_term: 1 leader_uuid: "b6648a2023f04a1b925af4b9221bd80a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } } }
I20260812 14:22:14.447253 2255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:22:14.449386 2271 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6648a2023f04a1b925af4b9221bd80a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } } }
I20260812 14:22:14.449467 2271 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:14.450021 2255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6648a2023f04a1b925af4b9221bd80a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } } }
I20260812 14:22:14.450103 2255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:22:14.450515 2295 mvcc.cc:205] Tried to move back new op lower bound from 7317686413087051776 to 7317686413073903616. Current Snapshot: MvccSnapshot[applied={T|T < 7317686413087051776}]
I20260812 14:22:14.451719 2286 catalog_manager.cc:1347] Generated new cluster ID: a744ea69595f4a8aa7c49f441b623467
I20260812 14:22:14.451766 2286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:22:14.457492 2154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } attempt: 3
I20260812 14:22:14.458540 2076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.443793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33402 (local address 127.8.85.62:45129)
0812 14:22:14.443872 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.443874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.444337 (+ 463us) server_negotiation.cc:407] Connection header received
0812 14:22:14.444456 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.444457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.444468 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.444499 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.446312 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.446998 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.449982 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.450274 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.450317 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.453600 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:14.453601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:14.453601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:14.453608 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:14.455083 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.457472 (+ 2389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.457473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.457473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.457530 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.457716 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.457717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.457718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.457832 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:14.457930 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:14.458377 (+ 447us) server_negotiation.cc:299] Negotiation successful
0812 14:22:14.458455 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 14:22:14.459766 2160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.443834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36794 (local address 127.8.85.61:34303)
0812 14:22:14.443905 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.443907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.445409 (+ 1502us) server_negotiation.cc:407] Connection header received
0812 14:22:14.445517 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.445518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.445530 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.445561 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.448134 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.448861 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.451949 (+ 3088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.452243 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.452279 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.455504 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:14.455505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:14.455505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:14.455513 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:14.457090 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.458094 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.458095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.458095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.458161 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.459139 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.459140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.459141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.459283 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:14.459353 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:14.459602 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 14:22:14.459676 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 14:22:14 dist-test-slave-5k54 krb5kdc[1982](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544534, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.61@KRBTEST.COM for kudu/127.8.85.60@KRBTEST.COM
I20260812 14:22:14.475728 2158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.457757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:43325 (local address 127.0.0.1:51816)
0812 14:22:14.457845 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:14.457856 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:14.457895 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:14.467669 (+ 9774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:14.467670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:14.467843 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:14.467889 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.467891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.468885 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.468886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.469723 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:14.469726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.470255 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.470256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:14.470314 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.470721 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:14.470726 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:14.472238 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:14.474612 (+ 2374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.474613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.474613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.474800 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.475023 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:14.475024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:14.475024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:14.475096 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:14.475405 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:14.475406 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:14.475491 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 14:22:14.475549 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:22:14.475633 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 14:22:14.476028 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.457970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51816 (local address 127.8.85.60:43325)
0812 14:22:14.467495 (+ 9525us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.467498 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.467510 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:22:14.467533 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.467534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.467555 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.467966 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.468002 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.468783 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.469825 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.470166 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.470481 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.472335 (+ 1854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:14.472337 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:14.472337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:14.472344 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:14.474519 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.474884 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.474885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.474885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.474952 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.475157 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.475158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.475158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.475274 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:14.475329 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:14.475851 (+ 522us) server_negotiation.cc:299] Negotiation successful
0812 14:22:14.475934 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9503,"thread_start_us":83,"threads_started":1}
I20260812 14:22:14.477421 2154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065: Bootstrap starting.
I20260812 14:22:14.478121 2154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065: Neither blocks nor log segments found. Creating new log.
I20260812 14:22:14.478399 2154 log.cc:824] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065: Log is configured to *not* fsync() on all Append() calls
I20260812 14:22:14.479200 2154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065: No bootstrap required, opened a new log
I20260812 14:22:14.481174 2154 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } }
I20260812 14:22:14.481333 2154 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4825d7fd37740448b4f90eaab1cf065, State: Initialized, Role: FOLLOWER
I20260812 14:22:14.481432 2154 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } }
I20260812 14:22:14.481750 2154 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:22:14.482115 2127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6648a2023f04a1b925af4b9221bd80a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f4825d7fd37740448b4f90eaab1cf065"
I20260812 14:22:14.482226 2127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:22:14.482247 2253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:14.444072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51808 (local address 127.8.85.60:43325)
0812 14:22:14.444113 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.444115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.450141 (+ 6026us) server_negotiation.cc:407] Connection header received
0812 14:22:14.450161 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.450162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.450170 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.450218 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.468219 (+ 18001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.468889 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.470256 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:14.470803 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:14.470843 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:14.473594 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:14.473595 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:14.473595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:14.473601 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:14.475683 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.481386 (+ 5703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.481387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.481387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.481459 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:14.481664 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:14.481665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:14.481666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:14.481781 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:14.481873 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:14.482081 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 14:22:14.482155 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:22:14.483139 2127 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6648a2023f04a1b925af4b9221bd80a in term 1.
I20260812 14:22:14.483345 2126 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065 [term 1 FOLLOWER]: Refusing update from remote peer b6648a2023f04a1b925af4b9221bd80a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:22:14.483641 2284 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:22:14.483947 2286 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:22:14.484639 2286 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:22:14.485847 2316 mvcc.cc:205] Tried to move back new op lower bound from 7317686413234089984 to 7317686413073903616. Current Snapshot: MvccSnapshot[applied={T|T < 7317686413087051776}]
I20260812 14:22:14.486093 2318 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6648a2023f04a1b925af4b9221bd80a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } } }
I20260812 14:22:14.486181 2318 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:22:14.486234 2318 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6648a2023f04a1b925af4b9221bd80a. Latest consensus state: current_term: 1 leader_uuid: "b6648a2023f04a1b925af4b9221bd80a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } } }
I20260812 14:22:14.486282 2318 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:22:14.486317 2318 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6648a2023f04a1b925af4b9221bd80a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } } }
I20260812 14:22:14.486390 2318 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:22:14.487401 2319 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:22:14.487691 2319 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:22:14.491600 2271 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6648a2023f04a1b925af4b9221bd80a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6648a2023f04a1b925af4b9221bd80a" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 45129 } } peers { permanent_uuid: "f4825d7fd37740448b4f90eaab1cf065" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 34303 } } peers { permanent_uuid: "912d8da8a65c46cd8ed3399f4b6b18f9" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 43325 } } }
I20260812 14:22:14.491694 2271 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:14.495644 2286 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6648a2023f04a1b925af4b9221bd80a: Generated new TSK 0
I20260812 14:22:14.495848 2286 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 14:22:14.524415 2253 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:14.522971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51826 (local address 127.8.85.60:43325)
0812 14:22:14.523019 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.523021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.523750 (+ 729us) server_negotiation.cc:407] Connection header received
0812 14:22:14.523857 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.523858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.523871 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.523903 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.524346 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51826: BlockingRecv error: recv got EOF from 127.0.0.1:51826 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 14:22:14.525254 2076 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:14.523461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33406 (local address 127.8.85.62:45129)
0812 14:22:14.523502 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.523504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.524723 (+ 1219us) server_negotiation.cc:407] Connection header received
0812 14:22:14.524843 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.524844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.524855 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.524884 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.525190 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33406: BlockingRecv error: recv got EOF from 127.0.0.1:33406 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 14:22:14.526384 2160 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:14.525704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36798 (local address 127.8.85.61:34303)
0812 14:22:14.525760 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.525763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.525777 (+ 14us) server_negotiation.cc:407] Connection header received
0812 14:22:14.525862 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.525863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.525875 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.525963 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.526299 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36798: BlockingRecv error: recv got EOF from 127.0.0.1:36798 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 14:22:14.528380 2160 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:14.527606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36812 (local address 127.8.85.61:34303)
0812 14:22:14.527642 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.527643 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.527859 (+ 216us) server_negotiation.cc:407] Connection header received
0812 14:22:14.527964 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.527965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.527982 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.528010 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.528336 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36812: BlockingRecv error: recv got EOF from 127.0.0.1:36812 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 14:22:14.531363 2160 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:14.530696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.8.85.61:34303)
0812 14:22:14.530742 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.530744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.530871 (+ 127us) server_negotiation.cc:407] Connection header received
0812 14:22:14.530963 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.530963 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.530973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.531000 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.531321 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36814: BlockingRecv error: recv got EOF from 127.0.0.1:36814 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 14:22:14.535207 2160 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:14.534507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36824 (local address 127.8.85.61:34303)
0812 14:22:14.534551 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.534553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.534748 (+ 195us) server_negotiation.cc:407] Connection header received
0812 14:22:14.534840 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.534841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.534850 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.534876 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.535163 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36824: BlockingRecv error: recv got EOF from 127.0.0.1:36824 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 14:22:14.541082 2160 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:14.540384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36826 (local address 127.8.85.61:34303)
0812 14:22:14.540416 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.540418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.540614 (+ 196us) server_negotiation.cc:407] Connection header received
0812 14:22:14.540709 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.540710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.540719 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.540746 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.541041 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36826: BlockingRecv error: recv got EOF from 127.0.0.1:36826 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 14:22:14.550987 2160 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:14.550238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36840 (local address 127.8.85.61:34303)
0812 14:22:14.550273 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.550285 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.550493 (+ 208us) server_negotiation.cc:407] Connection header received
0812 14:22:14.550587 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.550588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.550598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.550627 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.550933 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36840: BlockingRecv error: recv got EOF from 127.0.0.1:36840 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 14:22:14.569235 2160 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:14.568363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36844 (local address 127.8.85.61:34303)
0812 14:22:14.568405 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.568407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.568661 (+ 254us) server_negotiation.cc:407] Connection header received
0812 14:22:14.568767 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.568768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.568779 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.568811 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.569187 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36844: BlockingRecv error: recv got EOF from 127.0.0.1:36844 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 14:22:14.603755 2160 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:14.602683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36860 (local address 127.8.85.61:34303)
0812 14:22:14.602731 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.602735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.603095 (+ 360us) server_negotiation.cc:407] Connection header received
0812 14:22:14.603206 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.603208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.603221 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.603254 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.603692 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36860: BlockingRecv error: recv got EOF from 127.0.0.1:36860 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 14:22:14.670171 2160 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:14.669403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36868 (local address 127.8.85.61:34303)
0812 14:22:14.669470 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.669473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.669597 (+ 124us) server_negotiation.cc:407] Connection header received
0812 14:22:14.669652 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.669653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.669664 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.669717 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.670100 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36868: BlockingRecv error: recv got EOF from 127.0.0.1:36868 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 14:22:14.800424 2160 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:14.799631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36870 (local address 127.8.85.61:34303)
0812 14:22:14.799698 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:14.799701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:14.799828 (+ 127us) server_negotiation.cc:407] Connection header received
0812 14:22:14.799931 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:14.799932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:14.799944 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:14.799998 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:14.800354 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36870: BlockingRecv error: recv got EOF from 127.0.0.1:36870 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 14:22:15.058487 2160 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:15.057690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36880 (local address 127.8.85.61:34303)
0812 14:22:15.057744 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:15.057746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:15.057860 (+ 114us) server_negotiation.cc:407] Connection header received
0812 14:22:15.057984 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:15.057985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:15.057998 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:15.058071 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:15.058423 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36880: BlockingRecv error: recv got EOF from 127.0.0.1:36880 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:22:15.413421 2268 catalog_manager.cc:1259] Loaded cluster ID: a744ea69595f4a8aa7c49f441b623467
I20260812 14:22:15.413534 2268 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9: loading cluster ID for follower catalog manager: success
I20260812 14:22:15.414394 2268 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9: acquiring CA information for follower catalog manager: success
I20260812 14:22:15.414783 2268 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 912d8da8a65c46cd8ed3399f4b6b18f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:22:15.488194 2319 catalog_manager.cc:1259] Loaded cluster ID: a744ea69595f4a8aa7c49f441b623467
I20260812 14:22:15.488305 2319 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065: loading cluster ID for follower catalog manager: success
I20260812 14:22:15.489457 2319 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065: acquiring CA information for follower catalog manager: success
I20260812 14:22:15.489830 2319 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f4825d7fd37740448b4f90eaab1cf065: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:22:15.574222 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:15.571689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.8.85.61:34303)
0812 14:22:15.571826 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:15.571828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:15.571841 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:22:15.571934 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:15.571935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:15.571989 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:15.572068 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:22:15.572210 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:15.572712 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:15.573481 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:15.573677 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:15.573733 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:15.573883 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:22:15.573992 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:22:15.574052 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:15.574081 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 14:22:15.574098 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
I20260812 14:22:15.578521 2407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:15.576221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51836 (local address 127.8.85.60:43325)
0812 14:22:15.576353 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:15.576356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:15.576363 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:22:15.576399 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:15.576399 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:15.576435 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:15.576503 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:22:15.576616 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:15.577108 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:15.577783 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:15.578007 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:15.578061 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:15.578201 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:22:15.578296 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:22:15.578355 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:15.578392 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 14:22:15.578408 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260812 14:22:15.583457 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:15.580253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36896 (local address 127.8.85.61:34303)
0812 14:22:15.580293 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:15.580295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:15.580538 (+ 243us) server_negotiation.cc:407] Connection header received
0812 14:22:15.580630 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:15.580632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:15.580643 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:15.580700 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:22:15.580820 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:15.581609 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:15.582627 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:15.582939 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:15.582979 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:15.583197 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:22:15.583233 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:22:15.583285 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:15.583333 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 14:22:15.583354 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:22:15.583782 2407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:15.580273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51848 (local address 127.8.85.60:43325)
0812 14:22:15.580312 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:15.580313 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:15.580538 (+ 225us) server_negotiation.cc:407] Connection header received
0812 14:22:15.580752 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:15.580753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:15.580763 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:15.580810 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:22:15.580992 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:15.581597 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:15.582784 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:15.583036 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:15.583110 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:15.583249 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:22:15.583285 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:22:15.583350 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:15.583679 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 14:22:15.583701 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:22:15.583878 2415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:15.580247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33412 (local address 127.8.85.62:45129)
0812 14:22:15.580460 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:15.580462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:15.580520 (+ 58us) server_negotiation.cc:407] Connection header received
0812 14:22:15.580601 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:15.580602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:15.580655 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:15.580735 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:22:15.580930 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:15.581690 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:15.582749 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:15.583066 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:15.583126 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:15.583391 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:22:15.583525 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:22:15.583586 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:15.583745 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 14:22:15.583772 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":83,"threads_started":1}
I20260812 14:22:15.585603 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 1996
I20260812 14:22:15.594448 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 2079
I20260812 14:22:15.601485 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 2163
2026-08-12T14:22:15Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2909 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:22:15 dist-test-slave-5k54 krb5kdc[2422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:22:15 dist-test-slave-5k54 krb5kdc[2422](info): set up 2 sockets
Aug 12 14:22:15 dist-test-slave-5k54 krb5kdc[2422](info): commencing operation
krb5kdc: starting...
W20260812 14:22:16.717423 8532 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/test-user.keytab.
Aug 12 14:22:16 dist-test-slave-5k54 krb5kdc[2422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:22:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:22:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.85.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
Entry for principal kudu/127.8.85.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab.
I20260812 14:22:16.843214 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:43877
--webserver_interface=127.8.85.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:43877,127.8.85.61:33201,127.8.85.60:36185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:22:16.931605 2438 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:22:16.931843 2438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:22:16.931874 2438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:22:16.931919 2438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:22:16.933068 2438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:22:16.933132 2438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:22:16.933158 2438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:22:16.933188 2438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:22:16.933220 2438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:22:16.934759 2438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:43877,127.8.85.61:33201,127.8.85.60:36185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.62.keytab
--principal=kudu/127.8.85.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.62:43877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.85.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:22:16.935036 2438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:22:16.935276 2438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:22:16.940126 2446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:16.940174 2443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:16.940126 2444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:16.940382 2438 server_base.cc:1034] running on GCE node
I20260812 14:22:16.940620 2438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:22:16.940927 2438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:22:16.942122 2438 hybrid_clock.cc:648] HybridClock initialized: now 1786544536942097 us; error 36 us; skew 500 ppm
Aug 12 14:22:16 dist-test-slave-5k54 krb5kdc[2422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544536, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:22:16.944907 2438 init.cc:377] Logged in from keytab as kudu/127.8.85.62@KRBTEST.COM (short username kudu)
I20260812 14:22:16.945407 2438 webserver.cc:460] Webserver started at http://127.8.85.62:32931/ using document root <none> and password file <none>
I20260812 14:22:16.945606 2438 fs_manager.cc:359] Metadata directory not provided
I20260812 14:22:16.945652 2438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:22:16.945747 2438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:22:16.946863 2438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/data/instance:
uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7"
format_stamp: "Formatted at 2026-08-12 14:22:16 on dist-test-slave-5k54"
server_key: "3371f7ab0d31e1e48390c60e653149a8"
server_key_iv: "9e51a4bd401476dc243d96854f55c75c"
server_key_version: "encryptionkey@0"
I20260812 14:22:16.947193 2438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/wal/instance:
uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7"
format_stamp: "Formatted at 2026-08-12 14:22:16 on dist-test-slave-5k54"
server_key: "3371f7ab0d31e1e48390c60e653149a8"
server_key_iv: "9e51a4bd401476dc243d96854f55c75c"
server_key_version: "encryptionkey@0"
I20260812 14:22:16.948699 2438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:22:16.949494 2454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:16.949687 2438 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:22:16.949796 2438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/wal
uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7"
format_stamp: "Formatted at 2026-08-12 14:22:16 on dist-test-slave-5k54"
server_key: "3371f7ab0d31e1e48390c60e653149a8"
server_key_iv: "9e51a4bd401476dc243d96854f55c75c"
server_key_version: "encryptionkey@0"
I20260812 14:22:16.949884 2438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:22:16.966603 2438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:22:16.968787 2438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:22:16.968967 2438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:22:16.973539 2438 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.62:43877
I20260812 14:22:16.973574 2511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.62:43877 every 8 connection(s)
I20260812 14:22:16.973944 2438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/data/info.pb
I20260812 14:22:16.974948 2512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:22:16.976529 2512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:22:16.979399 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 2438
I20260812 14:22:16.979542 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-0/wal/instance
I20260812 14:22:16.979735 8532 external_mini_cluster.cc:1428] Setting key 195bdd81271bcbcea9baec244f1b6382
Aug 12 14:22:16 dist-test-slave-5k54 krb5kdc[2422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544536, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:22:16.987763 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:16.977292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38114 (local address 127.8.85.62:43877)
0812 14:22:16.977513 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:16.977516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:16.977539 (+ 23us) server_negotiation.cc:407] Connection header received
0812 14:22:16.978094 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:16.978095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:16.978112 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:16.978186 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:16.978771 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:16.979923 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:16.980855 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:16.981351 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:16.981481 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:16.984213 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:16.984221 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:16.984221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:16.984229 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:16.986063 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:16.986599 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:16.986602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:16.986602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:16.986678 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:16.986948 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:16.986949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:16.986949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:16.987143 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:16.987204 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:16.987503 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 14:22:16.987597 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":67,"threads_started":1}
I20260812 14:22:16.987856 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:16.977239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:43877 (local address 127.0.0.1:38114)
0812 14:22:16.977437 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:16.977476 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:16.977956 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:16.978226 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:16.978229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:16.978504 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:16.978682 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:16.978686 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:16.980021 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:16.980022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:16.980763 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:16.980765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:16.981488 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:16.981490 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:16.981586 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:16.982112 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:16.982123 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:16.984123 (+ 2000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:16.986164 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:16.986172 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:16.986175 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:16.986501 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:16.986747 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:16.986748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:16.986748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:16.986874 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:16.987214 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:16.987222 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:16.987312 (+ 90us) client_negotiation.cc:769] Sending connection context
0812 14:22:16.987450 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 14:22:16.987710 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":89,"thread_start_us":87,"threads_started":1}
I20260812 14:22:16.988632 2512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:22:16.989003 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:16.988862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33201 (local address 127.0.0.1:51990)
0812 14:22:16.988917 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:16.988975 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 14:22:16.989158 2457 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.85.61:33201: connect: Connection refused (error 111)
I20260812 14:22:16.989624 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:16.989507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33201 (local address 127.0.0.1:52002)
0812 14:22:16.989572 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:16.989601 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 14:22:16.989742 2512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.61:33201: Network error: Client connection negotiation failed: client connection to 127.8.85.61:33201: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.8.85.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.61.keytab.
Entry for principal kudu/127.8.85.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.61.keytab.
I20260812 14:22:17.011484 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.61:33201
--webserver_interface=127.8.85.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.61.keytab
--principal=kudu/127.8.85.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:43877,127.8.85.61:33201,127.8.85.60:36185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:22:17.038978 2512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } attempt: 1
I20260812 14:22:17.039530 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.039354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33201 (local address 127.0.0.1:52006)
0812 14:22:17.039452 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.039496 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 14:22:17.039947 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.039812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33201 (local address 127.0.0.1:52018)
0812 14:22:17.039900 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.039926 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260812 14:22:17.040047 2512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.61:33201: Network error: Client connection negotiation failed: client connection to 127.8.85.61:33201: connect: Connection refused (error 111)
W20260812 14:22:17.094992 2520 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:22:17.095193 2520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:22:17.095213 2520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:22:17.095232 2520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:22:17.096444 2520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:22:17.096487 2520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:22:17.096501 2520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:22:17.096514 2520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:22:17.096526 2520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:22:17.098006 2520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:43877,127.8.85.61:33201,127.8.85.60:36185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.61.keytab
--principal=kudu/127.8.85.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.61:33201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.85.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:22:17.098250 2520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:22:17.098475 2520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:22:17.103313 2529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:17.103394 2527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:17.103451 2520 server_base.cc:1034] running on GCE node
W20260812 14:22:17.103338 2526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:17.103789 2520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:22:17.104087 2520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:22:17.105234 2520 hybrid_clock.cc:648] HybridClock initialized: now 1786544537105209 us; error 41 us; skew 500 ppm
Aug 12 14:22:17 dist-test-slave-5k54 krb5kdc[2422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544537, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:22:17.107746 2520 init.cc:377] Logged in from keytab as kudu/127.8.85.61@KRBTEST.COM (short username kudu)
I20260812 14:22:17.108214 2520 webserver.cc:460] Webserver started at http://127.8.85.61:35471/ using document root <none> and password file <none>
I20260812 14:22:17.108187 2512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } attempt: 2
I20260812 14:22:17.108520 2520 fs_manager.cc:359] Metadata directory not provided
I20260812 14:22:17.108596 2520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:22:17.108712 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.108544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33201 (local address 127.0.0.1:52024)
0812 14:22:17.108634 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.108678 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 14:22:17.108745 2520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:22:17.109158 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.109090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33201 (local address 127.0.0.1:52030)
0812 14:22:17.109116 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.109141 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.61:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260812 14:22:17.109279 2512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.61:33201: Network error: Client connection negotiation failed: client connection to 127.8.85.61:33201: connect: Connection refused (error 111)
I20260812 14:22:17.110002 2520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/data/instance:
uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c"
format_stamp: "Formatted at 2026-08-12 14:22:17 on dist-test-slave-5k54"
server_key: "10f16d9d171d9894302b6f0d8f410402"
server_key_iv: "207ec8401c85acc0dd7d470f3f8df03b"
server_key_version: "encryptionkey@0"
I20260812 14:22:17.110378 2520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/wal/instance:
uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c"
format_stamp: "Formatted at 2026-08-12 14:22:17 on dist-test-slave-5k54"
server_key: "10f16d9d171d9894302b6f0d8f410402"
server_key_iv: "207ec8401c85acc0dd7d470f3f8df03b"
server_key_version: "encryptionkey@0"
I20260812 14:22:17.111792 2520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:22:17.112556 2537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:17.112732 2520 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:22:17.112805 2520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/wal
uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c"
format_stamp: "Formatted at 2026-08-12 14:22:17 on dist-test-slave-5k54"
server_key: "10f16d9d171d9894302b6f0d8f410402"
server_key_iv: "207ec8401c85acc0dd7d470f3f8df03b"
server_key_version: "encryptionkey@0"
I20260812 14:22:17.112891 2520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:22:17.124753 2520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:22:17.126801 2520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:22:17.126955 2520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:22:17.131242 2520 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.61:33201
I20260812 14:22:17.131326 2595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.61:33201 every 8 connection(s)
I20260812 14:22:17.131698 2520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/data/info.pb
I20260812 14:22:17.132774 2596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:22:17.134508 2596 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:22:17.136758 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 2520
I20260812 14:22:17.136871 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-1/wal/instance
I20260812 14:22:17.137080 8532 external_mini_cluster.cc:1428] Setting key 3adb47b73d37b2be1a014527a56b2e28
Aug 12 14:22:17 dist-test-slave-5k54 krb5kdc[2422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544537, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.61@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:22:17.144416 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.135329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38122 (local address 127.8.85.62:43877)
0812 14:22:17.135483 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.135485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.135617 (+ 132us) server_negotiation.cc:407] Connection header received
0812 14:22:17.136077 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.136079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.136092 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.136150 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.136777 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.137533 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.138498 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.138839 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.138922 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.141437 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:17.141439 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:17.141439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:17.141447 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:17.142947 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.143382 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.143383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.143383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.143481 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.143759 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.143760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.143761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.143879 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:17.143962 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:17.144239 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 14:22:17.144326 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 14:22:17.144601 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.135289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:43877 (local address 127.0.0.1:38122)
0812 14:22:17.135483 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.135528 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:17.135992 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:17.136211 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:17.136214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:17.136491 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:17.136681 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.136686 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.137625 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.137626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.138409 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.138411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.138971 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.138973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.139134 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.139690 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:17.139702 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:17.141283 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:17.143051 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.143058 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.143060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.143302 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.143553 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.143554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.143554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.143665 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.143952 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:17.143959 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:17.144101 (+ 142us) client_negotiation.cc:769] Sending connection context
0812 14:22:17.144205 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 14:22:17.144360 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":92,"threads_started":1}
I20260812 14:22:17.145123 2596 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.85.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.85.60@KRBTEST.COM" created.
Aug 12 14:22:17 dist-test-slave-5k54 krb5kdc[2422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544537, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.61@KRBTEST.COM for kudu/127.8.85.61@KRBTEST.COM
I20260812 14:22:17.153213 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.145415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33201 (local address 127.0.0.1:52034)
0812 14:22:17.145456 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.145468 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:17.145532 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:17.145820 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:17.145821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:17.145970 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:17.146018 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.146019 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.146860 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.146861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.147357 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.147358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.147865 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.147866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.147939 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.148349 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:17.148354 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:17.149605 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:17.152017 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.152018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.152018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.152230 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.152477 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.152477 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.152477 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.152532 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.152876 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:17.152877 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:17.152926 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 14:22:17.153029 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 14:22:17.153116 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 14:22:17.153368 2601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.145450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52034 (local address 127.8.85.61:33201)
0812 14:22:17.145653 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.145676 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.145695 (+ 19us) server_negotiation.cc:407] Connection header received
0812 14:22:17.145715 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.145717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.145729 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.145821 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.146095 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.146766 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.147450 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.147771 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.147865 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.149709 (+ 1844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:17.149717 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:17.149718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:17.149727 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:17.151930 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.152323 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.152326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.152326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.152403 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.152600 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.152601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.152601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.152798 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:17.152892 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:17.153071 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 14:22:17.153192 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":172,"thread_start_us":82,"threads_started":1}
I20260812 14:22:17.154079 2596 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:22:17.154439 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.154328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:36185 (local address 127.0.0.1:59042)
0812 14:22:17.154373 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.154420 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260812 14:22:17.154603 2541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
I20260812 14:22:17.155017 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.154931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:36185 (local address 127.0.0.1:59054)
0812 14:22:17.154976 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.155001 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 14:22:17.155143 2596 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.60:36185: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.85.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.60.keytab.
Entry for principal kudu/127.8.85.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.60.keytab.
I20260812 14:22:17.166337 8532 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
/tmp/dist-test-taskYVQ8wa/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.60:36185
--webserver_interface=127.8.85.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.60.keytab
--principal=kudu/127.8.85.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.85.20:36525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.85.62:43877,127.8.85.61:33201,127.8.85.60:36185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:22:17.200433 2512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } attempt: 3
I20260812 14:22:17.204373 2596 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } attempt: 1
Aug 12 14:22:17 dist-test-slave-5k54 krb5kdc[2422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544536, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for kudu/127.8.85.61@KRBTEST.COM
I20260812 14:22:17.204816 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.204654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:36185 (local address 127.0.0.1:59062)
0812 14:22:17.204736 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.204776 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 14:22:17.205160 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.205064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:36185 (local address 127.0.0.1:59076)
0812 14:22:17.205112 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.205136 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 14:22:17.205307 2596 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.60:36185: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
I20260812 14:22:17.207962 2601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.200917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52046 (local address 127.8.85.61:33201)
0812 14:22:17.200993 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.200995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.201006 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:22:17.201034 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.201035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.201053 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.201118 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.201446 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.201946 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.202646 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.202867 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.202948 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.205236 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:17.205238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:17.205238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:17.205245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:17.206823 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.207202 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.207203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.207204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.207253 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.207495 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.207496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.207496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.207602 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:17.207653 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:17.207828 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 14:22:17.207911 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 14:22:17.208014 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.200803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33201 (local address 127.0.0.1:52046)
0812 14:22:17.200851 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.200863 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:17.200958 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:17.201127 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:17.201128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:17.201293 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:17.201359 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.201361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.202028 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.202030 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.202557 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.202559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.202945 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.202946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.203029 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.203539 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:17.203544 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:17.205131 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:17.206910 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.206911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.206912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.207128 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.207318 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.207319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.207319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.207387 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.207664 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:17.207665 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:17.207717 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 14:22:17.207811 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 14:22:17.207915 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 14:22:17.208447 2512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:22:17.208786 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.208679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:36185 (local address 127.0.0.1:59090)
0812 14:22:17.208728 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.208762 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 14:22:17.209142 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.209049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:36185 (local address 127.0.0.1:59092)
0812 14:22:17.209097 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.209123 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 14:22:17.209252 2512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.60:36185: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
I20260812 14:22:17.240402 2512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } attempt: 1
I20260812 14:22:17.240952 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.240778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:36185 (local address 127.0.0.1:59108)
0812 14:22:17.240876 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.240920 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 14:22:17.241432 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.241288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:36185 (local address 127.0.0.1:59122)
0812 14:22:17.241377 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.241410 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 14:22:17.241572 2512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.60:36185: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
W20260812 14:22:17.250952 2605 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:22:17.251155 2605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:22:17.251173 2605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:22:17.251192 2605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:22:17.252410 2605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:22:17.252452 2605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:22:17.252466 2605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:22:17.252503 2605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:22:17.252532 2605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:22:17.254123 2605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.85.20:36525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.85.62:43877,127.8.85.61:33201,127.8.85.60:36185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/krb5kdc/kudu_127.8.85.60.keytab
--principal=kudu/127.8.85.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.85.60:36185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.85.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 11:50:47 UTC on ccycloud.quasar-kkharb.root.comops.site
build id 4581990
I20260812 14:22:17.254384 2605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:22:17.254616 2605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:22:17.259569 2610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:22:17.259577 2611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:17.259603 2605 server_base.cc:1034] running on GCE node
W20260812 14:22:17.259598 2613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:22:17.260008 2605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:22:17.260301 2605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:22:17.261482 2605 hybrid_clock.cc:648] HybridClock initialized: now 1786544537261467 us; error 26 us; skew 500 ppm
Aug 12 14:22:17 dist-test-slave-5k54 krb5kdc[2422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544537, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:22:17.263919 2605 init.cc:377] Logged in from keytab as kudu/127.8.85.60@KRBTEST.COM (short username kudu)
I20260812 14:22:17.264356 2605 webserver.cc:460] Webserver started at http://127.8.85.60:33775/ using document root <none> and password file <none>
I20260812 14:22:17.264561 2605 fs_manager.cc:359] Metadata directory not provided
I20260812 14:22:17.264601 2605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:22:17.264691 2605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:22:17.265674 2605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/data/instance:
uuid: "bc9f8603efe943d4945dd43847520635"
format_stamp: "Formatted at 2026-08-12 14:22:17 on dist-test-slave-5k54"
server_key: "fd45e25fe3d3d0eb619b5f16cd21c61e"
server_key_iv: "88f7b12bc093449bed3e3fbf1ee34fca"
server_key_version: "encryptionkey@0"
I20260812 14:22:17.265974 2605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/wal/instance:
uuid: "bc9f8603efe943d4945dd43847520635"
format_stamp: "Formatted at 2026-08-12 14:22:17 on dist-test-slave-5k54"
server_key: "fd45e25fe3d3d0eb619b5f16cd21c61e"
server_key_iv: "88f7b12bc093449bed3e3fbf1ee34fca"
server_key_version: "encryptionkey@0"
I20260812 14:22:17.267210 2605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:22:17.267974 2621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:22:17.268194 2605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:22:17.268283 2605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/data,/tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/wal
uuid: "bc9f8603efe943d4945dd43847520635"
format_stamp: "Formatted at 2026-08-12 14:22:17 on dist-test-slave-5k54"
server_key: "fd45e25fe3d3d0eb619b5f16cd21c61e"
server_key_iv: "88f7b12bc093449bed3e3fbf1ee34fca"
server_key_version: "encryptionkey@0"
I20260812 14:22:17.268356 2605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:22:17.273460 2596 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } attempt: 2
I20260812 14:22:17.273849 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.273729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:36185 (local address 127.0.0.1:59126)
0812 14:22:17.273783 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.273822 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 14:22:17.274196 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.274100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:36185 (local address 127.0.0.1:59136)
0812 14:22:17.274146 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.274177 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 14:22:17.274283 2596 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.85.60:36185: Network error: Client connection negotiation failed: client connection to 127.8.85.60:36185: connect: Connection refused (error 111)
I20260812 14:22:17.285463 2605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:22:17.287276 2605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:22:17.287523 2605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:22:17.291294 2605 rpc_server.cc:307] RPC server started. Bound to: 127.8.85.60:36185
I20260812 14:22:17.291357 2680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.85.60:36185 every 8 connection(s)
I20260812 14:22:17.291731 2605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/data/info.pb
I20260812 14:22:17.292713 2681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:22:17.294194 2681 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:22:17 dist-test-slave-5k54 krb5kdc[2422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544537, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.60@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:22:17.301519 8532 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu as pid 2605
I20260812 14:22:17.301644 8532 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYVQ8wa/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786544263511571-8532-0/minicluster-data/master-2/wal/instance
I20260812 14:22:17.301877 8532 external_mini_cluster.cc:1428] Setting key d76fc875c9f9fac14bb1753ce70bec34
I20260812 14:22:17.303591 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.294888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38138 (local address 127.8.85.62:43877)
0812 14:22:17.294964 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.294967 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.295149 (+ 182us) server_negotiation.cc:407] Connection header received
0812 14:22:17.295674 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.295675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.295689 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.295763 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.296348 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.296838 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.297828 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.298045 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.298131 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.300244 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:17.300245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:17.300246 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:17.300253 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:17.301986 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.302479 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.302481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.302481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.302552 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.302872 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.302873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.302873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.303079 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:17.303164 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:17.303398 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 14:22:17.303505 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 14:22:17.303738 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.294845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.62:43877 (local address 127.0.0.1:38138)
0812 14:22:17.295031 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.295066 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:17.295560 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:17.295833 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:17.295837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:17.296079 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:17.296255 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.296258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.296961 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.296962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.297728 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.297730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.298153 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.298154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.298271 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.298675 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:17.298683 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:17.300157 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:17.302084 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.302091 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.302093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.302405 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.302619 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.302619 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.302620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.302755 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.303155 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:17.303159 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:17.303265 (+ 106us) client_negotiation.cc:769] Sending connection context
0812 14:22:17.303343 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 14:22:17.303528 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":77,"threads_started":1}
I20260812 14:22:17.304294 2681 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:22:17 dist-test-slave-5k54 krb5kdc[2422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.60@KRBTEST.COM
Aug 12 14:22:17 dist-test-slave-5k54 krb5kdc[2422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544537, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.60@KRBTEST.COM for kudu/127.8.85.61@KRBTEST.COM
I20260812 14:22:17.311178 2685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.302280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59138 (local address 127.8.85.60:36185)
0812 14:22:17.302461 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.302465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.302521 (+ 56us) server_negotiation.cc:407] Connection header received
0812 14:22:17.302581 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.302601 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.302616 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.302691 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.303485 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.304396 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.305482 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.305753 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.305841 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.308393 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:17.308402 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:17.308402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:17.308409 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:17.309982 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.310332 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.310335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.310336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.310398 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.310612 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.310613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.310613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.310759 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:17.310905 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:17.310987 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 14:22:17.311062 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260812 14:22:17.311543 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.304566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.61:33201 (local address 127.0.0.1:52048)
0812 14:22:17.304679 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.304693 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:17.304728 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:17.304878 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:17.304879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:17.305031 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:17.305084 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.305086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.305801 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.305802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.306314 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.306315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.306785 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.306785 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.306828 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.307233 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:17.307237 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:17.308744 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:17.310413 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.310414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.310415 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.310585 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.310872 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.310873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.310873 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.310934 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.311229 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:17.311230 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:17.311274 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 14:22:17.311347 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 14:22:17.311460 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260812 14:22:17.311573 2601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.304635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52048 (local address 127.8.85.61:33201)
0812 14:22:17.304690 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.304692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.304721 (+ 29us) server_negotiation.cc:407] Connection header received
0812 14:22:17.304797 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.304798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.304809 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.304863 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.305152 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.305730 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.306377 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.306701 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.306775 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.308823 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:17.308825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:17.308826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:17.308833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:17.310298 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.310719 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.310720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.310720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.310798 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.311007 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.311009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.311009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.311149 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:17.311225 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:17.311386 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 14:22:17.311480 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:22:17.311854 2681 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:22:17 dist-test-slave-5k54 krb5kdc[2422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544537, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.60@KRBTEST.COM for kudu/127.8.85.60@KRBTEST.COM
Aug 12 14:22:17 dist-test-slave-5k54 krb5kdc[2422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.62@KRBTEST.COM
I20260812 14:22:17.316743 2512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } attempt: 2
I20260812 14:22:17.318646 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.312055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:36185 (local address 127.0.0.1:59152)
0812 14:22:17.312094 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.312103 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:17.312135 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:17.312330 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:17.312331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:17.312437 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:17.312464 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.312465 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.313125 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.313126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.313677 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.313680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.314112 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.314113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.314169 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.314562 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:17.314566 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:17.315675 (+ 1109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:17.317418 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.317418 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.317419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.317572 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.317788 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.317789 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.317789 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.317843 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.318307 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:17.318308 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:17.318354 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 14:22:17.318442 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 14:22:17.318538 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 14:22:17.318701 2685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.312087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59152 (local address 127.8.85.60:36185)
0812 14:22:17.312227 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.312229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.312237 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:22:17.312254 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.312254 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.312264 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.312323 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.312535 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.313053 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.313755 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.314029 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.314078 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.315762 (+ 1684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:17.315764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:17.315764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:17.315771 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:17.317331 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.317663 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.317664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.317664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.317714 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.317934 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.317934 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.317935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.318232 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:17.318315 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:17.318464 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 14:22:17.318587 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":120}
I20260812 14:22:17.320331 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.311970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38142 (local address 127.8.85.62:43877)
0812 14:22:17.312008 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.312010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.312058 (+ 48us) server_negotiation.cc:407] Connection header received
0812 14:22:17.312120 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.312121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.312130 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.312184 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.312738 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.313241 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.313945 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.314186 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.314268 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.316958 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:17.316960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:17.316960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:17.316968 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:17.318802 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.319378 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.319379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.319379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.319474 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.319656 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.319657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.319657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.319810 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:17.320016 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:17.320144 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 14:22:17.320230 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:22:17.320768 2681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635: Bootstrap starting.
I20260812 14:22:17.321453 2681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635: Neither blocks nor log segments found. Creating new log.
I20260812 14:22:17.321760 2681 log.cc:824] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635: Log is configured to *not* fsync() on all Append() calls
I20260812 14:22:17.322647 2681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635: No bootstrap required, opened a new log
Aug 12 14:22:17 dist-test-slave-5k54 krb5kdc[2422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544536, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.62@KRBTEST.COM for kudu/127.8.85.60@KRBTEST.COM
I20260812 14:22:17.324819 2681 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } }
I20260812 14:22:17.324990 2681 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9f8603efe943d4945dd43847520635, State: Initialized, Role: FOLLOWER
I20260812 14:22:17.325074 2681 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } }
I20260812 14:22:17.325428 2681 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:22:17.325402 2689 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } } }
I20260812 14:22:17.325489 2689 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635 [sys.catalog]: This master's current role is: FOLLOWER
Aug 12 14:22:17 dist-test-slave-5k54 krb5kdc[2422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.85.61@KRBTEST.COM
I20260812 14:22:17.327236 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.316992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:36185 (local address 127.0.0.1:59166)
0812 14:22:17.317036 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.317046 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:17.317082 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:17.317345 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:17.317346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:17.317487 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:17.317517 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.317519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.318923 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.318925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.320472 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.320475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.322305 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.322306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.322369 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.322762 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:17.322767 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:17.324163 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:17.326109 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.326110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.326111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.326346 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.326529 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.326529 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.326530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.326592 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.326943 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:17.326944 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:17.326991 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 14:22:17.327035 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:22:17.327127 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 14:22:17.327625 2686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.317035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59166 (local address 127.8.85.60:36185)
0812 14:22:17.317197 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.317198 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.317205 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:22:17.317223 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.317224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.317233 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.317324 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.317589 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.318828 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.322039 (+ 3211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.322225 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.322305 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.324255 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:17.324256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:17.324256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:17.324263 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:17.326020 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.326412 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.326413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.326413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.326472 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.326664 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.326665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.326665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.326844 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:17.327352 (+ 508us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:17.327409 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 14:22:17.327517 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":103,"threads_started":1}
W20260812 14:22:17.329139 2704 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:22:17.329193 2704 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:22:17.329232 2512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7: Bootstrap starting.
I20260812 14:22:17.329480 2601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.321008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52064 (local address 127.8.85.61:33201)
0812 14:22:17.321048 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.321050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.321059 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:22:17.321126 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.321127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.321135 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.321179 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.321649 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.322288 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.323240 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.323577 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.323656 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.326331 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:17.326332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:17.326333 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:17.326346 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:17.328033 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.328412 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.328414 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.328414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.328474 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.328660 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.328661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.328662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.328791 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:17.328952 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:17.329326 (+ 374us) server_negotiation.cc:299] Negotiation successful
0812 14:22:17.329400 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:22:17.329886 2512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7: Neither blocks nor log segments found. Creating new log.
I20260812 14:22:17.330128 2512 log.cc:824] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7: Log is configured to *not* fsync() on all Append() calls
I20260812 14:22:17.328549 2605 master_runner.cc:186] Error getting master registration for 127.8.85.62:43877: OK, instance_id { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" instance_seqno: 1786544536972279 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:22:17.330790 2605 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:22:17.331022 2512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7: No bootstrap required, opened a new log
I20260812 14:22:17.332615 8532 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:22:17.333498 2484 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38142
I20260812 14:22:17.334029 2512 raft_consensus.cc:348] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } }
I20260812 14:22:17.334160 2512 raft_consensus.cc:729] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 057ec0f4d5f54fb9bf7e3b677cf64fd7, State: Initialized, Role: FOLLOWER
I20260812 14:22:17.334229 2512 consensus_queue.cc:260] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } }
I20260812 14:22:17.334481 2709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } } }
I20260812 14:22:17.334550 2512 sys_catalog.cc:564] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:22:17.334582 2484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 14:22:17.334658 2484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:22:17.335680 2484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } }
I20260812 14:22:17.336241 2651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bc9f8603efe943d4945dd43847520635"
I20260812 14:22:17.336364 2651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:22:17.335974 2484 leader_election.cc:288] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [CANDIDATE]: Term 1 election: Requested vote from peers a0969e64ec3b4d8d87e0fc9fff01d41c (127.8.85.61:33201), bc9f8603efe943d4945dd43847520635 (127.8.85.60:36185)
I20260812 14:22:17.337328 2651 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 057ec0f4d5f54fb9bf7e3b677cf64fd7 in term 1.
I20260812 14:22:17.337589 2459 leader_election.cc:302] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 057ec0f4d5f54fb9bf7e3b677cf64fd7, bc9f8603efe943d4945dd43847520635; no voters:
I20260812 14:22:17.334553 2709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:22:17.337827 2709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:22:17.337889 2709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [term 1 LEADER]: Becoming Leader. State: Replica: 057ec0f4d5f54fb9bf7e3b677cf64fd7, State: Running, Role: LEADER
I20260812 14:22:17.337973 2709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } }
I20260812 14:22:17.338390 2709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 057ec0f4d5f54fb9bf7e3b677cf64fd7. Latest consensus state: current_term: 1 leader_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } } }
I20260812 14:22:17.338472 2709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:17.338043 2438 master_runner.cc:186] Error getting master registration for 127.8.85.61:33201: OK, instance_id { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" instance_seqno: 1786544537130050 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:22:17.340791 2438 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 14:22:17.344256 2720 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:22:17.344352 2720 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:22:17.344455 2728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:22:17.344623 2728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:22:17.348855 2686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.339129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59180 (local address 127.8.85.60:36185)
0812 14:22:17.339170 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.339172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.339713 (+ 541us) server_negotiation.cc:407] Connection header received
0812 14:22:17.339810 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.339811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.339822 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.339853 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.341493 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.342238 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.343018 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.343321 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.343380 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.345870 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:17.345872 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:17.345872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:17.345879 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:17.347580 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.347996 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.347998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.347998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.348054 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.348245 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.348246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.348246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.348379 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:17.348470 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:17.348672 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 14:22:17.348753 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:22:17.349923 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.338760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38146 (local address 127.8.85.62:43877)
0812 14:22:17.338798 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.338800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.339342 (+ 542us) server_negotiation.cc:407] Connection header received
0812 14:22:17.339486 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.339487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.339499 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.339545 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.340880 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.341606 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.342616 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.342944 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.343025 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.346408 (+ 3383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:17.346409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:17.346410 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:17.346417 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:17.348248 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.348899 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.348901 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.348901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.348962 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.349335 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.349336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.349336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.349470 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:17.349579 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:17.349739 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 14:22:17.349823 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:22:17.351145 2601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.339384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52078 (local address 127.8.85.61:33201)
0812 14:22:17.339484 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.339485 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.340190 (+ 705us) server_negotiation.cc:407] Connection header received
0812 14:22:17.340737 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.340738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.340748 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.340779 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.343057 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.344223 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.345298 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.345654 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.345696 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.348096 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:17.348098 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:17.348098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:17.348106 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:17.349835 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.350381 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.350382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.350383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.350437 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.350646 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.350647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.350647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.350756 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:17.350876 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:17.350965 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 14:22:17.351035 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:22:17.360447 2651 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635 [term 1 FOLLOWER]: Refusing update from remote peer 057ec0f4d5f54fb9bf7e3b677cf64fd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:22:17.360772 2709 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:22:17.362102 2689 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 057ec0f4d5f54fb9bf7e3b677cf64fd7. Latest consensus state: current_term: 1 leader_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } } }
I20260812 14:22:17.362198 2689 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:22:17.365413 2596 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } attempt: 3
I20260812 14:22:17.368379 2726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } } }
I20260812 14:22:17.368503 2726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:17.368582 2733 mvcc.cc:205] Tried to move back new op lower bound from 7317686425025228800 to 7317686424937017344. Current Snapshot: MvccSnapshot[applied={T|T < 7317686425025228800}]
I20260812 14:22:17.368558 2689 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } } }
I20260812 14:22:17.368629 2689 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:22:17.368782 2735 mvcc.cc:205] Tried to move back new op lower bound from 7317686425025228800 to 7317686424937017344. Current Snapshot: MvccSnapshot[applied={T|T < 7317686425025228800}]
Aug 12 14:22:17 dist-test-slave-5k54 krb5kdc[2422](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786544537, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.85.61@KRBTEST.COM for kudu/127.8.85.60@KRBTEST.COM
I20260812 14:22:17.374336 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.365669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.85.60:36185 (local address 127.0.0.1:59194)
0812 14:22:17.365716 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 14:22:17.365728 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:22:17.365765 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:22:17.365906 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:22:17.365907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:22:17.366090 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:22:17.366134 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.366136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.367328 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.367329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.368125 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:22:17.368128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.368785 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.368787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:22:17.368868 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.369300 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:22:17.369306 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:22:17.370822 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:22:17.373058 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.373059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.373060 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.373278 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.373566 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:22:17.373567 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:22:17.373567 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:22:17.373625 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:22:17.374032 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:22:17.374033 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:22:17.374088 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 14:22:17.374144 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 14:22:17.374237 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 14:22:17.375058 2686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:17.365730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59194 (local address 127.8.85.60:36185)
0812 14:22:17.365776 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.365777 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.365787 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:22:17.365821 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.365822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.365831 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.365894 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.366521 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.367255 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.368242 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:17.368551 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:17.368609 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:17.371166 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:22:17.371168 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:22:17.371168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:22:17.371174 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:22:17.372952 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.373370 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.373371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.373371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.373452 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:22:17.373699 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:22:17.373700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:22:17.373701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:22:17.373909 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:22:17.374654 (+ 745us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:17.374731 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 14:22:17.374927 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:22:17.375867 2728 catalog_manager.cc:1347] Generated new cluster ID: c59c1dcbfd564de989e48d292f71162b
I20260812 14:22:17.375914 2728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:22:17.376772 2596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c: Bootstrap starting.
I20260812 14:22:17.377480 2596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c: Neither blocks nor log segments found. Creating new log.
I20260812 14:22:17.377761 2596 log.cc:824] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:22:17.378608 2596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c: No bootstrap required, opened a new log
I20260812 14:22:17.380781 2596 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } }
I20260812 14:22:17.380915 2596 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0969e64ec3b4d8d87e0fc9fff01d41c, State: Initialized, Role: FOLLOWER
I20260812 14:22:17.380992 2596 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } }
I20260812 14:22:17.381364 2567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c"
I20260812 14:22:17.381467 2566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:22:17.381297 2596 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:22:17.382334 2566 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c [term 1 FOLLOWER]: Refusing update from remote peer 057ec0f4d5f54fb9bf7e3b677cf64fd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:22:17.382647 2709 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:22:17.381292 2740 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } } }
I20260812 14:22:17.383091 2740 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:22:17.383888 2567 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 057ec0f4d5f54fb9bf7e3b677cf64fd7 in term 1.
I20260812 14:22:17.385850 2740 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 057ec0f4d5f54fb9bf7e3b677cf64fd7. Latest consensus state: current_term: 1 leader_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } } }
I20260812 14:22:17.385931 2740 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:22:17.385963 2740 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } } }
I20260812 14:22:17.385996 2740 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:22:17.388000 2751 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:22:17.389096 2751 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:22:17.388825 2752 mvcc.cc:205] Tried to move back new op lower bound from 7317686425025228800 to 7317686424937017344. Current Snapshot: MvccSnapshot[applied={T|T < 7317686425025228800}]
I20260812 14:22:17.390174 2709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "057ec0f4d5f54fb9bf7e3b677cf64fd7" member_type: VOTER last_known_addr { host: "127.8.85.62" port: 43877 } } peers { permanent_uuid: "a0969e64ec3b4d8d87e0fc9fff01d41c" member_type: VOTER last_known_addr { host: "127.8.85.61" port: 33201 } } peers { permanent_uuid: "bc9f8603efe943d4945dd43847520635" member_type: VOTER last_known_addr { host: "127.8.85.60" port: 36185 } } }
I20260812 14:22:17.390269 2709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7 [sys.catalog]: This master's current role is: LEADER
I20260812 14:22:17.391645 2728 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:22:17.392529 2728 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:22:17.400262 2728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 057ec0f4d5f54fb9bf7e3b677cf64fd7: Generated new TSK 0
I20260812 14:22:17.400485 2728 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 14:22:17.431164 2515 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:17.429490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38148 (local address 127.8.85.62:43877)
0812 14:22:17.429538 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.429540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.430492 (+ 952us) server_negotiation.cc:407] Connection header received
0812 14:22:17.430599 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.430600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.430612 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.430642 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.431096 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38148: BlockingRecv error: recv got EOF from 127.0.0.1:38148 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 14:22:17.432083 2686 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:17.429860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59202 (local address 127.8.85.60:36185)
0812 14:22:17.429904 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.429906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.431387 (+ 1481us) server_negotiation.cc:407] Connection header received
0812 14:22:17.431638 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.431639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.431650 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.431680 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.432022 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59202: BlockingRecv error: recv got EOF from 127.0.0.1:59202 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 14:22:17.432686 2601 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:17.430209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52086 (local address 127.8.85.61:33201)
0812 14:22:17.430250 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.430252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.431591 (+ 1339us) server_negotiation.cc:407] Connection header received
0812 14:22:17.432284 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.432285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.432296 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.432325 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.432629 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52086: BlockingRecv error: recv got EOF from 127.0.0.1:52086 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 14:22:17.435158 2601 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:17.434407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52100 (local address 127.8.85.61:33201)
0812 14:22:17.434440 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.434441 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.434631 (+ 190us) server_negotiation.cc:407] Connection header received
0812 14:22:17.434735 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.434736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.434753 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.434782 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.435113 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52100: BlockingRecv error: recv got EOF from 127.0.0.1:52100 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 14:22:17.438149 2601 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:17.437400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52106 (local address 127.8.85.61:33201)
0812 14:22:17.437431 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.437433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.437653 (+ 220us) server_negotiation.cc:407] Connection header received
0812 14:22:17.437746 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.437747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.437758 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.437786 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.438107 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52106: BlockingRecv error: recv got EOF from 127.0.0.1:52106 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 14:22:17.441977 2601 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:17.441223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52116 (local address 127.8.85.61:33201)
0812 14:22:17.441255 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.441257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.441470 (+ 213us) server_negotiation.cc:407] Connection header received
0812 14:22:17.441563 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.441564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.441573 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.441601 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.441939 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52116: BlockingRecv error: recv got EOF from 127.0.0.1:52116 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 14:22:17.447860 2601 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:17.447072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52118 (local address 127.8.85.61:33201)
0812 14:22:17.447104 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.447106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.447321 (+ 215us) server_negotiation.cc:407] Connection header received
0812 14:22:17.447418 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.447419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.447465 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.447497 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.447822 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52118: BlockingRecv error: recv got EOF from 127.0.0.1:52118 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 14:22:17.457984 2601 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:17.457158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52120 (local address 127.8.85.61:33201)
0812 14:22:17.457193 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.457195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.457433 (+ 238us) server_negotiation.cc:407] Connection header received
0812 14:22:17.457533 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.457534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.457545 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.457574 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.457931 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52120: BlockingRecv error: recv got EOF from 127.0.0.1:52120 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 14:22:17.476248 2601 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:17.475282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52132 (local address 127.8.85.61:33201)
0812 14:22:17.475325 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.475327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.475628 (+ 301us) server_negotiation.cc:407] Connection header received
0812 14:22:17.475736 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.475737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.475749 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.475792 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.476192 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52132: BlockingRecv error: recv got EOF from 127.0.0.1:52132 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 14:22:17.510524 2601 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:17.509654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52140 (local address 127.8.85.61:33201)
0812 14:22:17.509703 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.509713 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.509897 (+ 184us) server_negotiation.cc:407] Connection header received
0812 14:22:17.509980 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.509981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.510000 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.510062 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.510447 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52140: BlockingRecv error: recv got EOF from 127.0.0.1:52140 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 14:22:17.576598 2601 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:17.575790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52144 (local address 127.8.85.61:33201)
0812 14:22:17.575908 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.575911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.575996 (+ 85us) server_negotiation.cc:407] Connection header received
0812 14:22:17.576062 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.576063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.576075 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.576154 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.576533 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52144: BlockingRecv error: recv got EOF from 127.0.0.1:52144 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260812 14:22:17.706734 2601 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:17.705985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52146 (local address 127.8.85.61:33201)
0812 14:22:17.706065 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.706067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.706160 (+ 93us) server_negotiation.cc:407] Connection header received
0812 14:22:17.706211 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.706212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.706224 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.706277 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.706654 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52146: BlockingRecv error: recv got EOF from 127.0.0.1:52146 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 14:22:17.964874 2601 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:22:17.964125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52152 (local address 127.8.85.61:33201)
0812 14:22:17.964197 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:17.964199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:17.964212 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:22:17.964323 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:17.964325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:17.964336 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:17.964399 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:22:17.964799 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52152: BlockingRecv error: recv got EOF from 127.0.0.1:52152 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 14:22:18.329640 2704 catalog_manager.cc:1259] Loaded cluster ID: c59c1dcbfd564de989e48d292f71162b
I20260812 14:22:18.329723 2704 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635: loading cluster ID for follower catalog manager: success
I20260812 14:22:18.330485 2704 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635: acquiring CA information for follower catalog manager: success
I20260812 14:22:18.330753 2704 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bc9f8603efe943d4945dd43847520635: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:22:18.389662 2751 catalog_manager.cc:1259] Loaded cluster ID: c59c1dcbfd564de989e48d292f71162b
I20260812 14:22:18.389752 2751 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c: loading cluster ID for follower catalog manager: success
I20260812 14:22:18.390599 2751 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c: acquiring CA information for follower catalog manager: success
I20260812 14:22:18.390882 2751 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a0969e64ec3b4d8d87e0fc9fff01d41c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:22:18.481112 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:18.478390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52168 (local address 127.8.85.61:33201)
0812 14:22:18.478522 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:18.478524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:18.478531 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:22:18.478556 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:18.478557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:18.478610 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:18.478725 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:22:18.478893 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:18.479413 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:18.480258 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:18.480468 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:18.480551 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:18.480746 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:22:18.480865 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:22:18.480959 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:18.480982 (+ 23us) server_negotiation.cc:299] Negotiation successful
0812 14:22:18.481005 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":53,"threads_started":1}
I20260812 14:22:18.486012 2840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:18.483327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59214 (local address 127.8.85.60:36185)
0812 14:22:18.483475 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:18.483478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:18.483489 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:22:18.483545 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:18.483546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:18.483601 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:18.483689 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:22:18.483839 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:18.484371 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:18.485178 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:18.485397 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:18.485477 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:18.485658 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:22:18.485767 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:22:18.485855 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:18.485893 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 14:22:18.485915 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":66,"threads_started":1}
I20260812 14:22:18.490724 2840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:18.487985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59224 (local address 127.8.85.60:36185)
0812 14:22:18.488026 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:18.488028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:18.488228 (+ 200us) server_negotiation.cc:407] Connection header received
0812 14:22:18.488322 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:18.488323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:18.488333 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:18.488383 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:22:18.488549 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:18.489239 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:18.489998 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:18.490272 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:18.490333 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:18.490462 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:22:18.490494 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:22:18.490543 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:18.490631 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 14:22:18.490650 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:22:18.491119 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:18.487780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52172 (local address 127.8.85.61:33201)
0812 14:22:18.487817 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:18.487819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:18.488137 (+ 318us) server_negotiation.cc:407] Connection header received
0812 14:22:18.488164 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:18.488165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:18.488178 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:18.488227 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:22:18.488400 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:18.489071 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:18.490155 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:18.490487 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:18.490542 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:18.490701 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:22:18.490730 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:22:18.490778 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:18.491017 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 14:22:18.491042 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:22:18.491171 2848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:22:18.487748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38162 (local address 127.8.85.62:43877)
0812 14:22:18.487850 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 14:22:18.487852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:22:18.487981 (+ 129us) server_negotiation.cc:407] Connection header received
0812 14:22:18.488066 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:22:18.488067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:22:18.488142 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:22:18.488234 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:22:18.488379 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:18.489147 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:18.490137 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:22:18.490469 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:22:18.490540 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:22:18.490717 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:22:18.490824 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:22:18.490886 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:22:18.491022 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 14:22:18.491049 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":50,"threads_started":1}
I20260812 14:22:18.492713 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 2438
I20260812 14:22:18.502499 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 2520
I20260812 14:22:18.508430 8532 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYVQ8wa/build/release/bin/kudu with pid 2605
2026-08-12T14:22:18Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2904 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5814 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (274997 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST